# ECSE 506: Stochastic Control and Decision Theory

## Aditya Mahajan

Winter 2022

### About | Lectures | Notes | Coursework

# 1 Preliminaries

**Lemma 1**-
Suppose that \(Q(z,w)\) is a quadratic function of vectors \(z\) and \(w\), positive definite in \(w\).

Let \(Q_{ww} = ∂^2 Q(z,w)/∂w^2\). Since \(Q(z,w)\) is a quadratic function, \(Q_{ww}\) does not depend on \(z\). Since \(Q\) is positive definite in \(w\), \(Q_{ww} > 0\).

Suppose \(w \in \reals^r\). Define \(q = \log[ (2π)^{r/2} \det(Q_{ww})^{-1/2}]\). Then, for a fixed value of \(z\) \[ \int \exp\bigl[ -Q(z,w)\bigr] dw = \exp\bigl[ q - \inf_{w \in \reals^r} Q(z,w) \bigr]. \]

The point of the lemma is that, if one replaces an integration with respect to \(w\) by a minimization of \(Q\) with respect to \(w\), then the result is correct as far as terms dependent on the second argument \(z\) are concerned.

#### Proof

For the fixed value of \(z\), let \(\hat w\) be the minimizing value of \(Q(z,w)\). Then, one can write

\[ Q(z,w) = Q(z, \hat w) + \tfrac 12 (w-\hat w)^\TRANS Q_{ww} (w - \hat w). \]

The result follows from substituting this in the left hand side of the expression in the Lemma and observing that (e.g., from the form of the density function of a multi-nominal Gaussian),

\[\begin{equation} \int \exp[ - \tfrac 12 (w - \hat w)^\TRANS Q_{ww} (w - \hat w) ] dw = \exp[-q]. \tag*{$\Box$} \end{equation}\]

An immediate implication of the above result is the following.

This entry was last updated on 13 Jun 2020 and posted in MDP and tagged linear systems, riccati equation, lqr, risk sensitive.