# 39 Stochastic stability

Consider a discrete-time stochastic system that starts from a constant initial condition \(x_0\) and evolves according to \[\begin{equation}\label{eq:stability-dynamics} X_{t+1} = f(X_t, W_{t}) \end{equation}\] where \(X_t \in \reals^{n}\), \(\{Y_t\}_{t \ge 1}\) is a \(\reals^d\)-valued stochastic process on a probabilty space \((Ω,\mathcal F, \PR)\). Define \(\mathcal F_t = σ(W_{1:t})\). Then \(\{X_t\}_{t \ge 1}\) is an \(\{\mathcal F_t\}_{t \ge 1}\) adapted process. In addition, if \(\{W_t\}_{t \ge 1}\) is an independent process, then \(\{X_t\}_{t \ge 1}\) is Markov.

A point \(x^*\) is said to be an *equilibrium* of \eqref{eq:stability-dynamics} if \(f(x^*,y) = x^*\) for all \(w \in \reals^d\). Without loss of generality, we assume that the origin \(x = 0\) is an equilibrium.

## 39.1 Different notions of stability

First we recall some terminology related to almost sure convergence.

A random sequence \(\{X_t\}_{t \ge 1}\) in a sample space \(Ω\) converges to a random variable \(X\)

*almost surely*if \[ \PR\Bigl(ω \in Ω : \lim_{t \to ∞} \| X_t(ω) - X(ω) \| = 0 \Bigr) = 1. \]The convergence is said to be exponentially fast with rate no slower than \(γ^{-1}\) for some \(γ > 1\) (not dependent on \(ω\)) if \(γ^t \| X_t - X \|\) converges almost surely to some \(Δ \ge 0\).

Given a set \(\mathcal D \in \reals^n\), a random sequence \(\{X_t\}_{t \ge 1}\) is said to converge to \(\mathcal D\) almost surely if \[ \PR\Bigl(ω \in Ω : \lim_{t \to ∞} \mathrm{dist}(X_t(ω), \mathcal D) = 0 \Bigr) = 1, \] where \(\mathrm{dist}(x,\mathcal D) = \inf_{\tilde x \in \mathcal D} \| x - \tilde x\|\).

**Definition 39.1** The origin of \eqref{eq:stability-dynamics} is said to be:

**Stable in probability**if \(\lim_{x_0 \to 0} \PR(\sup_{t \ge 1} \| X_t \| > ε ) = 0\) for any \(ε > 0\).**Asymptotically stable in prbability**if it is stable in probability and moreover \(\lim_{x_0 \to 0} \PR(\lim_{t \to ∞} \|X_t\| = 0) = 1\).**Exponentially stable in prbability**if for some \(γ > 1\) (not dependent on \(ω\)), \(\lim_{x_0 \to 0} \PR(\lim_{t \to ∞} \|γ^t X_t\| = 0) = 1\).

## 39.2 Sufficient conditions for stochastic stability

**Definition 39.2** Given a set \(\mathcal Q \in \reals^n\) containing the origin, the origin of \eqref{eq:stability-dynamics} is said to be:

**locally a.s. asymptotically stable in \(\mathcal Q\)**if starting from \(x_0 \in \mathcal Q\) all the sample paths \(X_t\) stay in \(\mathcal Q\) for all \(t \ge 1\) and converge to origin almost surely.**locally a.s. exponentially stable in \(\mathcal Q\)**if it is locally a.s. asymptotically stable and the convergence is exponentially fast.

If the above properties hold for \(\mathcal Q = \reals^n\), the system is said to be **globally** a.s. asymptotically (or exponentially) stable.

A continuous function \(h \colon [0, a) \to [0, ∞)\) is said to belong to class \(\mathcal K\) if it is strictly increasing and \(h(0) = 0\).

**Theorem 39.1** For the stochastic discrete-time system \eqref{eq:stability-dynamics}, let \(\{X_t\}_{t \ge 1}\) be Markov.

Let \(V \colon \reals^n \to \reals\) be a continuous positive definite and radially unbounded function. Define the level set \(\mathcal Q_{λ} \coloneqq \{ x : 0 \le V(x) < λ \}\) for some \(λ > 0\).

Let \(φ \colon \reals^n \to \reals\) be a continious function that satisfies \(φ(x) \ge 0\) for all \(x \in \mathcal Q_{λ}\).

Suppose the following property holds: for all \(x \in \reals^n\), \[ \EXP[ V(X_{t+1}) \mid X_t = x ] - V(x) \le -φ(x), \quad \forall t \ge 1. \] Then:

For any initial condition \(x_0 \in \mathcal Q_{λ}\), \(\{X_t\}_{t \ge 1}\) converges to \(\mathcal D_1 \coloneqq \{x \in \mathcal Q_{λ} : φ(x) = 0 \}\) with probability at least \(1 - V(x_0)/λ\).

if moreover \(φ(x)\) is positive definite on \(\mathcal Q_{λ}\) and there exist two calss \(\mathcal K\) functions \(h_1\) and \(h_2\) such that \(h_1(\|x\|) \le V(x) \le h_2(\|x\|)\), then \(x = 0\) is asymptotically stable in probability.

Under slightly stronger conditions, it is also possible to characterize the rate of convergence.

**Theorem 39.2** For the stochastic discrete-time system \eqref{eq:stability-dynamics}, let \(\{X_t\}_{t \ge 1}\) be Markov.

Let \(V \colon \reals^n \to \reals\) be a continous nonnegative function.

Suppose the following condition holds: there exists an \(α \in (0,1)\) such that \[ \EXP[ V(X_{t+1}) \mid X_t = x ] - V(x) \le -α V(x), \quad \forall t \ge 1. \] Then:

For any initial state \(x_0\), \(V(X_t)\) almost surely converges to \(0\) exponentially fast with a rate no slower than \(1-α\).

If moreover \(V\) satisfies \(c_1 \|x\|^p \le V(x) \le c_2 \|x\|^p\) for some \(c_1, c_2, p > 0\), then \(x = 0\) is globally a.s. exponentially stable.

## 39.3 Weaker sufficient conditions for stochastic stability

**Theorem 39.3** For the stochastic discrete-time system \eqref{eq:stability-dynamics}, let \(V \colon \reals^n \to \reals\) be a continuous nonnegative and radially unbounded function. Define the set \(Q_{λ} \coloneqq \{x : V(x) < λ \}\) for some \(λ > 0\).

Suppose the following conditions hold:

For any \(t\) such that \(X_t \in \mathcal Q_{λ}\), we have \[ \EXP[ V(X_{t+1}) \mid \mathcal F_t ] - V(X_t) \le 0. \]

There is an integer \(T \ge 1\) (not depent on \(ω\)) and a continous function \(φ \colon \reals^n \to \reals\) that satisfies \(φ(x) \ge 0\) for all \(x \in \mathcal Q_{λ}\) such that for any \(t\), \[ \EXP[ V(X_{t+\color{red}{T}}) \mid \mathcal F_t ] - V(X_t) \le -φ(X_t) \]

Then, the implications of Theorem 39.1 hold.

The sufficient conditions for exponential stability can be weakened in a similar manner.

**Theorem 39.4** Suppose assumptions a) and b) of Theorem 39.3 are satisfied with the inequality of b) strengthened to \[ \EXP[ V(X_{t+\color{red}{T}}) \mid \mathcal F_t ] - V(X_t) \le -αV(X_t) \] for some \(α \in (0,1)\).

Then, the implications of Theorem 39.2 hold.

## Notes

The material in this section is adapted from Qin et al. (2020).