15  Brownian motion

Author

Affiliation

Aditya Mahajan

McGill University

In this lecture, we introduce Brownian motion, the continuous-time analogue of the Gaussian random walk. It is the fundamental continuous-time Gaussian process with stationary independent increments. As with the Poisson process, a process with independent increments can be described through the distributions of those increments.

15.1 Gaussian increments and Brownian motion

1. In 1827, the botanist Robert Brown observed that small particles suspended in a fluid undergo a highly irregular motion. In 1905, Einstein explained this motion as the cumulative effect of many collisions with the molecules of the fluid. His model related observable particle displacements to molecular parameters and provided a way to estimate Avogadro’s number. Wiener gave the first rigorous mathematical construction of the process in 1923; for this reason, Brownian motion is also called a Wiener process.

2. The probabilistic model is motivated by the following considerations. The displacement over a sufficiently long interval is the cumulative effect of many small collisions, so the central limit theorem suggests a Gaussian distribution. It is also reasonable to assume that

   • displacements over intervals of the same length have the same distribution;
   • displacements over disjoint intervals are independent;
   • there is no systematic drift; and
   • the particle position changes continuously.

3. A standard Brownian motion is a stochastic process \(\{W(t)\}_{t \ge 0}\) satisfying

   • \(W(0) = 0\);
   • for \(0 \le s < t\), \[ W(t) - W(s) \sim \mathcal N(0, t - s); \]
   • increments over disjoint time intervals are independent; and
   • the sample paths \(t \mapsto W(t)\) are continuous.

   Thus, Brownian motion has stationary independent increments. The variance \(t - s\) fixes the time scale; more generally, \(σ W(t)\) has increment variance \(σ^2(t - s)\).

4. The plot below shows one approximate sample path. It is obtained by using many short independent Gaussian increments and joining the resulting points. Brownian sample paths are continuous, but they remain irregular at every time scale.

function gaussianRandom(mean = 0, stdev = 1) {
  const u = 1 - Math.random()
  const v = Math.random()
  const z = Math.sqrt(-2 * Math.log(u)) * Math.cos(2 * Math.PI * v)
  return mean + stdev * z
}

viewof rerun_brownian = Inputs.button("Generate sample path")

brownian_path = {
  rerun_brownian
  const T = 1
  const N = 1000
  const dt = T / N
  const points = new Array(N + 1)
  let W = 0
  points[0] = {t: 0, W_t: 0}
  for (let n = 1; n <= N; n++) {
    W += gaussianRandom(0, Math.sqrt(dt))
    points[n] = {t: n * dt, W_t: W}
  }
  return points
}

Plot.plot({
  grid: true,
  height: 180,
  x: {domain: [0, 1], label: "t"},
  y: {label: "W(t)"},
  marks: [
    Plot.ruleY([0]),
    Plot.line(brownian_path, {x: "t", y: "W_t", strokeWidth: 1.5})
  ]
})

5. For any \(0 = t_0 < t_1 < \cdots < t_k\), the vector \[ \MATRIX{W(t_1) & W(t_2) & \cdots & W(t_k)} \] is a linear transformation of the independent Gaussian increments \[ W(t_1), \quad W(t_2) - W(t_1), \quad \ldots, \quad W(t_k) - W(t_{k-1}). \] It is therefore jointly Gaussian. Hence \(\{W(t)\}_{t \ge 0}\) is a Gaussian process.

15.2 Mean, auto-covariance, and stationarity

1. Since each increment has zero mean, \[ μ^W(t) = \EXP[W(t)] = 0. \]

2. Suppose \(0 \le s \le t\). Write \[ W(t) = W(s) + \bigl(W(t) - W(s)\bigr). \] The two terms on the right are independent, and the second has zero mean. Therefore, \[\begin{align*} R^W(s, t) &= \COV(W(s), W(t)) \\ &= \VAR(W(s)) \\ &= s. \end{align*}\] By symmetry, \[ R^W(s, t) = \min\{s, t\}. \] Since the mean is zero, the auto-correlation and auto-covariance are the same.

3. Brownian motion is not stationary: its variance \[ \VAR(W(t)) = t \] depends on time. What is stationary is its increment law. For every \(h \ge 0\), \[ W(t + h) - W(t) \sim \mathcal N(0, h), \] independently of \(t\). For a fixed \(h\), the Gaussian process \(\{V_h(t)\}_{t \ge 0}\) given by \[ V_h(t) = W(t + h) - W(t) \] is stationary, with auto-covariance \[ R^{V_h}(s, t) = \bigl(h - |t - s|\bigr)^+, \quad (x)^+ = \max\{x, 0\}. \]

15.3 White noise and the frequency-domain view

1. The derivative of Brownian motion does not exist as an ordinary random process. Nonetheless, write formally \[ w(t) = \frac{dW(t)}{dt}, \] meaning that integrals of \(w\) recover Brownian increments: \[ \int_a^b w(t) \, dt = W(b) - W(a). \] Treat \(w\) as zero-mean, so that a covariance of integrals is an expectation of a product. Expanding the product and passing the expectation inside (formally) gives \[\begin{align*} \COV\biggl(\int_a^b w(t)\,dt, \int_c^d w(s)\,ds\biggr) &= \EXP\Biggl[\biggl(\int_a^b w(t)\,dt\biggr)\biggl(\int_c^d w(s)\,ds\biggr)\Biggr] \\ &= \int_a^b \int_c^d \EXP\bigl[w(t)w(s)\bigr]\,dt\,ds \\ &= \int_a^b \int_c^d R^w(t - s)\,dt\,ds, \end{align*}\] where \(R^w(τ) = \COV(w(t + τ), w(t))\). Independent increments force the left-hand side to vanish whenever \((a, b]\) and \((c, d]\) are disjoint, so \(R^w(τ) = 0\) for \(τ \ne 0\). When \((c, d] = (a, b]\), the two terms on the left-hand-side are the same, so the covariance is the variance of the increment \(W(b) - W(a)\). Thus, \[ \VAR(W(b) - W(a)) = \int_a^b \int_a^b R^w(t - s)\,dt\,ds. \] An ordinary function that vanishes off \(τ = 0\) cannot produce this: the diagonal \((t = s)\) has area zero in the \((t, s)\)-plane. However, both constraints are satisfied by the Dirac delta, \[ R^w(τ) = δ(τ), \] because \(\int_a^b δ(t - s)\,ds\) equals \(1\) if \(t \in (a, b]\) and equals \(0\) otherwise.

   The corresponding power spectral density is \[ S_w(Ω) = \int_{-∞}^{∞} δ(τ) e^{-\mathsf j Ω τ} \, dτ = 1. \] Thus \(w\) has equal power at all frequencies. As in discrete time, a flat spectrum is why the process is called white noise. White noise of intensity \(q\) is the same object with \[ R^w(τ) = q δ(τ), \quad S_w(Ω) = q. \]

2. The discrete-time frequency-domain analysis extends formally to continuous time. If a continuous-time LTI system with frequency response \(H(\mathsf j Ω)\) is driven by a stationary input with power spectral density \(S_U(Ω)\), then the output has \[ S_Y(Ω) = |H(\mathsf j Ω)|^2 S_U(Ω). \] In particular, for white-noise input of intensity \(q\), \[ S_Y(Ω) = q |H(\mathsf j Ω)|^2. \] These relations apply to general wide-sense stationary processes under the usual stability and integrability assumptions. They do not give an ordinary power spectral density for Brownian motion itself, since Brownian motion is not stationary.

15.4 Continuous-time state-space models

1. By analogy with the discrete-time state-space model, a first attempt is to write \[ \frac{dX(t)}{dt} = A X(t) + B w(t), \] where \(w(t)\) is white noise. This is not an ordinary differential equation: white noise is not an ordinary function of time, and the resulting state process generally has no derivative in the conventional sense.

2. Instead, the model is written as a stochastic differential equation \[ dX(t) = A X(t) \, dt + B \, dW(t). \] Its integral interpretation is \[ X(t) = X(0) + \int_0^t A X(s) \, ds + \int_0^t B \, dW(s). \] The last term is a stochastic integral and cannot be defined using ordinary Riemann integration. The systematic theory used to define such integrals and manipulate stochastic differential equations is called Itô calculus.

3. One reason ordinary calculus does not apply is the scale of Brownian increments: \[ W(t + h) - W(t) = O(\sqrt h). \] Consequently, the square of an increment is of order \(h\), rather than a higher-order negligible term. More precisely, the quadratic variation of Brownian motion satisfies \[ \sum_k \bigl(W(t_{k+1}) - W(t_k)\bigr)^2 \longrightarrow t \] as the partition of \([0, t]\) becomes finer. Thus, second-order terms in a Taylor expansion contribute to the stochastic chain rule. This is the source of the additional second-derivative term in Itô’s formula.

4. Stochastic integration and Itô calculus are not developed here. Continuous-time Gaussian state-space models extend the discrete-time Gauss–Markov models studied earlier, but that extension requires new mathematical tools.