viewof p_bernoulli_process = Inputs.range([0, 1], {label: "p", value: 0.3, step: 0.05})
viewof rerun_bernoulli_process = Inputs.button("Generate sample path")
points_bernoulli_process = {
rerun_bernoulli_process
const T = 120
const points = new Array(T)
let count = 0
for (let n = 1; n <= T; n++) {
const B_n = (Math.random() <= p_bernoulli_process) ? 1 : 0
count += B_n
points[n-1] = {n: n, B_n: B_n, N_n: count}
}
return points
}10 Stochastic processes
10.1 Processes and sample paths
A stochastic process is a family of random variables indexed by time. It may be viewed as a random signal.
A discrete-time stochastic process is a collection \(\{X_n\}_{n\in\mathcal T}\), where the time index set \(\mathcal T\) is typically \(\integers_{\ge0}\), \(\naturalnumbers\), or \(\integers\). All the random variables are defined on a common probability space \((Ω,\ALPHABET F,\PR)\).
For each fixed time \(n\), \(X_n\) is a random variable. For each fixed \(ω\inΩ\), the sequence \[ X_0(ω),X_1(ω),X_2(ω),\ldots \] is called a realization or sample path of the process. Thus, a process can be viewed either as a sequence of random variables or as a random choice of a signal.
A continuous-time stochastic process is a collection \(\{X_t\}_{t\in\mathcal T}\), where \(\mathcal T\) is typically \([0,∞)\) or \(\reals\). Continuous-time processes require some additional technical care. We will focus primarily on discrete-time processes.
10.2 Examples
Example 10.1 (Bernoulli process and Bernoulli counting process) Let \(\{B_n\}_{n\ge1}\) be an i.i.d. sequence with \[ B_n\sim\operatorname{Bernoulli}(p), \] and define the counting process \[ N_0=0, \qquad N_n=\sum_{k=1}^n B_k, \quad n\ge1. \] The process \(\{B_n\}_{n\ge1}\) is called a Bernoulli process, and \(\{N_n\}_{n\ge0}\) is called the associated Bernoulli counting process. The latter is also sometimes called a binomial process.
The plots below show the Bernoulli process and its counting process for the same realization.
Some properties of these processes are as follows:
- At every time \(n\), \(B_n\sim\operatorname{Bernoulli}(p)\), while \[ N_n\sim\operatorname{Binomial}(n,p). \]
- The random variables \(B_1,B_2,\ldots\) are independent, but \(N_1,N_2,\ldots\) are not. In fact, \(N_{n+1}=N_n+B_{n+1}\).
- For every \(n\ge0\) and \(h\ge1\), \[ N_{n+h}-N_n=B_{n+1}+\cdots+B_{n+h} \sim\operatorname{Binomial}(h,p). \] Thus, the distribution of an increment depends on its length \(h\) but not on its starting time \(n\). Moreover, increments over disjoint time intervals are independent.
More generally, a process \(\{X_n\}_{n\ge0}\) has stationary increments if the distribution of \(X_{n+h}-X_n\) depends only on the length \(h\) and not on the starting time \(n\). It has independent increments if, for every \(0\le n_0<n_1<\cdots<n_r\), the random variables \[ X_{n_1}-X_{n_0},\ X_{n_2}-X_{n_1},\ldots, X_{n_r}-X_{n_{r-1}} \] are independent. Thus, the Bernoulli counting process has stationary independent increments.
Example 10.2 (Signal plus noise) Consider the process \[ Y_n=x_n+W_n, \] where \(x_n=\sin(2πn/25)\) is a deterministic discrete-time signal and \(\{W_n\}_{n\ge0}\) is an i.i.d. Gaussian sequence with \(W_n\sim\mathcal N(0,σ^2)\) and \(σ=0.2\). A sample path of \(\{Y_n\}_{n\ge0}\) is shown below.
At each fixed time, \[ Y_n\sim\mathcal N(x_n,σ^2). \] A sample path looks like a noisy signal, while the random variable \(Y_n\) describes uncertainty at one particular time.
10.3 Finite-dimensional distributions and dependence
The marginal distribution of \(X_n\) at each time is not enough to specify a stochastic process. For example, consider the following two processes: \[ X_n=B_n, \qquad \widetilde X_n=B_1, \quad n\ge1, \] where \(\{B_n\}_{n\ge1}\) is an i.i.d. Bernoulli process. Both \(X_n\) and \(\widetilde X_n\) have a \(\operatorname{Bernoulli}(p)\) distribution at every time. However, the variables \(X_1,X_2,\ldots\) are independent, while \(\widetilde X_1=\widetilde X_2=\cdots\).
A stochastic process is specified by its finite-dimensional distributions: for every finite collection of time indices \(n_1,\ldots,n_k\), we specify the joint distribution of \[ (X_{n_1},\ldots,X_{n_k}). \]
For a discrete-valued process, the general chain rule gives \[\begin{align*} &\PR(X_0=x_0,X_1=x_1,\ldots,X_n=x_n)\\ &\quad=\PR(X_0=x_0) \PR(X_1=x_1\mid X_0=x_0) \PR(X_2=x_2\mid X_0=x_0,X_1=x_1)\\ &\qquad\mathrel{\phantom{=}}\cdots \PR(X_n=x_n\mid X_0=x_0,\ldots,X_{n-1}=x_{n-1}). \end{align*}\] Thus, a process can be described by specifying how the conditional distribution of the next random variable depends on the past.
For an i.i.d. process, the past has no influence on the next random variable. A Markov chain permits temporal dependence but assumes that the current value summarizes all the relevant information from the past: \[ \PR(X_{n+1}=x_{n+1}\mid X_0=x_0,\ldots,X_n=x_n) =\PR(X_{n+1}=x_{n+1}\mid X_n=x_n). \] Markov chains are studied in detail in the next chapter.
10.4 Gaussian processes
A stochastic process \(\{X_n\}_{n\ge0}\) is called a second-order process if \[ \EXP[|X_n|^2]<∞ \qquad\text{for every }n. \] For such a process, the mean function and auto-covariance function \[ μ^X_n=\EXP[X_n], \qquad R^X_{n,m}=\COV(X_n,X_m) \] are well-defined. Second-order processes can be studied through these first and second moments even when their complete finite-dimensional distributions are unknown.
A stochastic process \(\{X_n\}_{n\ge0}\) is a Gaussian process if, for every finite collection of time indices \(n_1,\ldots,n_k\), the random vector \[ \MATRIX{X_{n_1}&\cdots&X_{n_k}}^\TRANS \] has a multivariate Gaussian distribution. Equivalently, every finite linear combination of the random variables \(X_n\) is Gaussian.
Every Gaussian process is a second-order process. Moreover, unlike a general second-order process, a Gaussian process is completely characterized by its mean function and auto-covariance function: \[ μ^X_n = \EXP[X_n], \qquad R^X_{n,m} = \COV(X_n,X_m). \] In particular, for any \(n_1,\ldots,n_k\), these functions specify the mean vector and covariance matrix of the corresponding Gaussian random vector.
When two processes \(\{X_n\}_{n \ge 0}\) and \(\{Y_n\}_{n \ge 0}\) are defined on a common probability space and every finite collection drawn from either process is jointly Gaussian, we say that the processes are jointly Gaussian. Their second-order structure is then described by the auto-covariances \(R^X_{n,m}\), \(R^Y_{n,m}\) and the cross-covariance \[ R^{XY}_{n,m} = \COV(X_n, Y_m), \] which satisfies \(R^{XY}_{n,m} = R^{YX}_{m,n}\).
A zero-mean Gaussian process \(\{W_n\}_{n\ge0}\) is called white Gaussian noise with variance \(σ_W^2\) if \[ R^W_{n,m} = \COV(W_n,W_m) = σ_W^2\IND\{n = m\}. \] Equivalently, white Gaussian noise is an i.i.d. sequence of Gaussian random variables with \[ W_n \sim \mathcal N(0,σ_W^2). \] We call the process standard white Gaussian noise when \(σ_W^2 = 1\).
Example 10.3 (Cosine with Gaussian amplitude) Let \[ X_n = A \cos(Ω n) + B \sin(Ω n), \] where \(Ω \in \reals\) is fixed and \(A,B\) are i.i.d. \(\mathcal N(0,σ^2)\). A sample path is shown below.
The mean is zero. Expanding \(\EXP[X_m X_n]\) and using independence of \(A\) and \(B\) gives \[ R^X_{m,n} = σ^2 \cos\bigl(Ω(m - n)\bigr) = σ^2 \cos\bigl(Ω|m - n|\bigr). \] Thus every finite collection is Gaussian (as a linear image of \((A,B)\)), so \(\{X_n\}_{n\ge0}\) is a Gaussian process, and the covariance depends only on the lag.
10.5 Stationarity
A discrete-time stochastic process \(\{X_n\}_{n\ge0}\) is strict-sense stationary if shifting all time indices by the same amount does not change any finite-dimensional distribution. More precisely, for every \(k\), every collection of time indices \(n_1,\ldots, n_k\), and every \(h\ge0\), the random vectors \[ (X_{n_1},\ldots,X_{n_k}) \quad\text{and}\quad (X_{n_1+h},\ldots,X_{n_k+h}) \] have the same distribution.
The Bernoulli process \(\{B_n\}_{n\ge1}\) is strictly stationary because it is i.i.d. The Bernoulli counting process \(\{N_n\}_{n\ge0}\) is not stationary because \(N_n\sim\operatorname{Binomial}(n,p)\) depends on \(n\). It does, however, have stationary independent increments.
The signal-plus-noise process in Example 10.2 is not stationary because \[ \EXP[Y_n]=x_n=\sin(2πn/25) \] depends on time.
Strict-sense stationarity implies that the marginal distribution of \(X_n\) does not depend on \(n\). Consequently, if the relevant moments exist, \(\EXP[X_n]\) and \(\VAR(X_n)\) are constant. The converse is not true: time-invariant marginal distributions, or even constant means and variances, do not by themselves imply stationarity.
For a Gaussian process, strict-sense stationarity is equivalent to the mean being constant and the auto-covariance depending only on the time difference. With the convention \[ R^X_k=\COV(X_{n+k},X_n), \] stationarity means that \(R^X_k\) does not depend on \(n\). The converse holds because the mean and covariance determine every finite-dimensional distribution of a Gaussian process. In particular, Example 10.3 is strict-sense stationary.
Stationarity is defined in terms of the distribution of the process, which is an ensemble property. In applications, however, we often observe only one sample path. Roughly speaking, a stationary process is ergodic if time averages along one sufficiently long sample path converge to the corresponding ensemble averages. For example, for zero-mean jointly stationary ergodic processes \(\{X_n\}\) and \(\{Y_n\}\), the cross-covariance can be estimated as \[ R^{XY}_k =\COV(X_{n+k},Y_n) \approx \frac{1}{N}\sum_{n=0}^{N-1}X_{n+k}Y_n \] for large \(N\). We will not develop the theory of ergodic processes in this course.
Important classes of stochastic processes use different structures to simplify the description of their finite-dimensional distributions:
- Processes with independent increments describe temporal evolution through the distributions of their increments.
- Markov chains describe dependence through an initial distribution and one-step conditional distributions, with the current state summarizing the relevant past.
- Gaussian processes are completely characterized by their mean and auto-covariance functions.
State-space Gauss–Markov models combine the Markov and Gaussian viewpoints for stochastic systems with inputs, outputs, and an internal state.