38 Stochastic stability
Consider a discrete-time stochastic system that starts from a constant initial condition \(x_0\) and evolves according to \[\begin{equation}\label{eq:stability-dynamics} X_{t+1} = f(X_t, W_{t}) \end{equation}\] where \(X_t \in \reals^{n}\), \(\{Y_t\}_{t \ge 1}\) is a \(\reals^d\)-valued stochastic process on a probabilty space \((Ω,\mathcal F, \PR)\). Define \(\mathcal F_t = σ(W_{1:t})\). Then \(\{X_t\}_{t \ge 1}\) is an \(\{\mathcal F_t\}_{t \ge 1}\) adapted process. In addition, if \(\{W_t\}_{t \ge 1}\) is an independent process, then \(\{X_t\}_{t \ge 1}\) is Markov.
A point \(x^*\) is said to be an equilibrium of \eqref{eq:stability-dynamics} if \(f(x^*,y) = x^*\) for all \(w \in \reals^d\). Without loss of generality, we assume that the origin \(x = 0\) is an equilibrium.
38.1 Different notions of stability
First we recall some terminology related to almost sure convergence.
A random sequence \(\{X_t\}_{t \ge 1}\) in a sample space \(Ω\) converges to a random variable \(X\) almost surely if \[ \PR\Bigl(ω \in Ω : \lim_{t \to ∞} \| X_t(ω) - X(ω) \| = 0 \Bigr) = 1. \]
The convergence is said to be exponentially fast with rate no slower than \(γ^{-1}\) for some \(γ > 1\) (not dependent on \(ω\)) if \(γ^t \| X_t - X \|\) converges almost surely to some \(Δ \ge 0\).
Given a set \(\mathcal D \in \reals^n\), a random sequence \(\{X_t\}_{t \ge 1}\) is said to converge to \(\mathcal D\) almost surely if \[ \PR\Bigl(ω \in Ω : \lim_{t \to ∞} \mathrm{dist}(X_t(ω), \mathcal D) = 0 \Bigr) = 1, \] where \(\mathrm{dist}(x,\mathcal D) = \inf_{\tilde x \in \mathcal D} \| x - \tilde x\|\).
Definition 38.1 The origin of \eqref{eq:stability-dynamics} is said to be:
Stable in probability if \(\lim_{x_0 \to 0} \PR(\sup_{t \ge 1} \| X_t \| > ε ) = 0\) for any \(ε > 0\).
Asymptotically stable in prbability if it is stable in probability and moreover \(\lim_{x_0 \to 0} \PR(\lim_{t \to ∞} \|X_t\| = 0) = 1\).
Exponentially stable in prbability if for some \(γ > 1\) (not dependent on \(ω\)), \(\lim_{x_0 \to 0} \PR(\lim_{t \to ∞} \|γ^t X_t\| = 0) = 1\).
38.2 Sufficient conditions for stochastic stability
Definition 38.2 Given a set \(\mathcal Q \in \reals^n\) containing the origin, the origin of \eqref{eq:stability-dynamics} is said to be:
locally a.s. asymptotically stable in \(\mathcal Q\) if starting from \(x_0 \in \mathcal Q\) all the sample paths \(X_t\) stay in \(\mathcal Q\) for all \(t \ge 1\) and converge to origin almost surely.
locally a.s. exponentially stable in \(\mathcal Q\) if it is locally a.s. asymptotically stable and the convergence is exponentially fast.
If the above properties hold for \(\mathcal Q = \reals^n\), the system is said to be globally a.s. asymptotically (or exponentially) stable.
A continuous function \(h \colon [0, a) \to [0, ∞)\) is said to belong to class \(\mathcal K\) if it is strictly increasing and \(h(0) = 0\).
Theorem 38.1 For the stochastic discrete-time system \eqref{eq:stability-dynamics}, let \(\{X_t\}_{t \ge 1}\) be Markov.
Let \(V \colon \reals^n \to \reals\) be a continuous positive definite and radially unbounded function. Define the level set \(\mathcal Q_{λ} \coloneqq \{ x : 0 \le V(x) < λ \}\) for some \(λ > 0\).
Let \(φ \colon \reals^n \to \reals\) be a continious function that satisfies \(φ(x) \ge 0\) for all \(x \in \mathcal Q_{λ}\).
Suppose the following property holds: for all \(x \in \reals^n\), \[ \EXP[ V(X_{t+1}) \mid X_t = x ] - V(x) \le -φ(x), \quad \forall t \ge 1. \] Then:
For any initial condition \(x_0 \in \mathcal Q_{λ}\), \(\{X_t\}_{t \ge 1}\) converges to \(\mathcal D_1 \coloneqq \{x \in \mathcal Q_{λ} : φ(x) = 0 \}\) with probability at least \(1 - V(x_0)/λ\).
if moreover \(φ(x)\) is positive definite on \(\mathcal Q_{λ}\) and there exist two calss \(\mathcal K\) functions \(h_1\) and \(h_2\) such that \(h_1(\|x\|) \le V(x) \le h_2(\|x\|)\), then \(x = 0\) is asymptotically stable in probability.
Under slightly stronger conditions, it is also possible to characterize the rate of convergence.
Theorem 38.2 For the stochastic discrete-time system \eqref{eq:stability-dynamics}, let \(\{X_t\}_{t \ge 1}\) be Markov.
Let \(V \colon \reals^n \to \reals\) be a continous nonnegative function.
Suppose the following condition holds: there exists an \(α \in (0,1)\) such that \[ \EXP[ V(X_{t+1}) \mid X_t = x ] - V(x) \le -α V(x), \quad \forall t \ge 1. \] Then:
For any initial state \(x_0\), \(V(X_t)\) almost surely converges to \(0\) exponentially fast with a rate no slower than \(1-α\).
If moreover \(V\) satisfies \(c_1 \|x\|^p \le V(x) \le c_2 \|x\|^p\) for some \(c_1, c_2, p > 0\), then \(x = 0\) is globally a.s. exponentially stable.
38.3 Weaker sufficient conditions for stochastic stability
Theorem 38.3 For the stochastic discrete-time system \eqref{eq:stability-dynamics}, let \(V \colon \reals^n \to \reals\) be a continuous nonnegative and radially unbounded function. Define the set \(Q_{λ} \coloneqq \{x : V(x) < λ \}\) for some \(λ > 0\).
Suppose the following conditions hold:
For any \(t\) such that \(X_t \in \mathcal Q_{λ}\), we have \[ \EXP[ V(X_{t+1}) \mid \mathcal F_t ] - V(X_t) \le 0. \]
There is an integer \(T \ge 1\) (not depent on \(ω\)) and a continous function \(φ \colon \reals^n \to \reals\) that satisfies \(φ(x) \ge 0\) for all \(x \in \mathcal Q_{λ}\) such that for any \(t\), \[ \EXP[ V(X_{t+\color{red}{T}}) \mid \mathcal F_t ] - V(X_t) \le -φ(X_t) \]
Then, the implications of Theorem 38.1 hold.
The sufficient conditions for exponential stability can be weakened in a similar manner.
Theorem 38.4 Suppose assumptions a) and b) of Theorem 38.3 are satisfied with the inequality of b) strengthened to \[ \EXP[ V(X_{t+\color{red}{T}}) \mid \mathcal F_t ] - V(X_t) \le -αV(X_t) \] for some \(α \in (0,1)\).
Then, the implications of Theorem 38.2 hold.
Notes
The material in this section is adapted from Qin et al. (2020).