Chapter 11 Probability cheatsheet

11.1 Gaussian distribution

Definition 11.1 (Multivariate Gaussian distribution) $$N\left(x;\mu,\Sigma\right)=\frac{1}{\left(2\pi\right)^{n/2}\left|\Sigma\right|^{1/2}}exp\left(-\frac{1}{2}\left(x-\mu\right)'\Sigma^{-1}\left(x-\mu\right)\right)$$

where $x,\mu \in \mathbb{R}^n$, $\Sigma \in \mathbb{R}^{n\times n}$

Linear combination $Ax$ of Gaussian $x$ is Gaussian with:

• mean: $A\mu$
• variance: $A\Sigma A'$

The conditional expectation of Gaussian with respect to $Ax=y$ is Gaussian with

• conditional mean: $\mu + \Sigma A'(A\Sigma A')^{-1}(y-A\mu)$
• conditional variance: $\Sigma - \Sigma A'(A\Sigma A')^{-1} A\Sigma$

As a result, one obtains the following general formula:

$$\begin{align*} p\left(Bx|Ax=y\right) & =N(Bx;B\mu+B\Sigma A'(A\Sigma A')^{-1}(y-A\mu),\\ & B\Sigma B'-B\Sigma A'(A\Sigma A')^{-1}A\Sigma B') \tag{11.1} \end{align*}$$

Note also that $\mathbb E(exp(\lambda X))=exp(\lambda\mu+\frac{\lambda^2\sigma^2}{2}).$