Chapter 11 Probability cheatsheet

11.1 Gaussian distribution

Definition 11.1 (Multivariate Gaussian distribution) \[ N\left(x;\mu,\Sigma\right)=\frac{1}{\left(2\pi\right)^{n/2}\left|\Sigma\right|^{1/2}}exp\left(-\frac{1}{2}\left(x-\mu\right)'\Sigma^{-1}\left(x-\mu\right)\right) \]

where \(x,\mu \in \mathbb{R}^n\), \(\Sigma \in \mathbb{R}^{n\times n}\)

Linear combination \(Ax\) of Gaussian \(x\) is Gaussian with:

• mean: \(A\mu\)
• variance: \(A\Sigma A'\)

The conditional expectation of Gaussian with respect to \(Ax=y\) is Gaussian with

• conditional mean: \(\mu + \Sigma A'(A\Sigma A')^{-1}(y-A\mu)\)
• conditional variance: \(\Sigma - \Sigma A'(A\Sigma A')^{-1} A\Sigma\)

As a result, one obtains the following general formula:

\[\begin{align*} p\left(Bx|Ax=y\right) & =N(Bx;B\mu+B\Sigma A'(A\Sigma A')^{-1}(y-A\mu),\\ & B\Sigma B'-B\Sigma A'(A\Sigma A')^{-1}A\Sigma B') \tag{11.1} \end{align*}\]

Note also that \(\mathbb E(exp(\lambda X))=exp(\lambda\mu+\frac{\lambda^2\sigma^2}{2}).\)