Appendix

The next theorem is a generalization of Fisher’s Theorem (Theorem 2.2). It is key in statistical inference.

Theorem 2.6 Let \(\boldsymbol{X}=(X_1,\ldots,X_n)'\) be a vector of iid rv’s distributed as \(\mathcal{N}(0,\sigma^2).\) We define the linear combinations

\[\begin{align*} Z_i=\boldsymbol{c}_i' \boldsymbol{X}, \quad i=1,\ldots,p\leq n-1, \end{align*}\]

where the vectors \(\boldsymbol{c}_i\in \mathbb{R}^n\) are orthonormal, that is

\[\begin{align*} \boldsymbol{c}_i'\boldsymbol{c}_j=\begin{cases} 0 & \text{if}\ i\neq j,\\ 1 & \text{if}\ i=j. \end{cases} \end{align*}\]

Then,

\[\begin{align*} Y=\boldsymbol{X}'\boldsymbol{X}-\sum_{i=1}^p Z_i^2 \end{align*}\]

is independent from \(Z_1,\ldots,Z_p\) and, in addition,

\[\begin{align*} \frac{Y}{\sigma^2}\sim \chi_{n-p}^2. \end{align*}\]

Proof (Proof of Theorem 2.6). Select \(n-p\) vectors \(\boldsymbol{c}_{p+1},\ldots,\boldsymbol{c}_n\) so that \(\{\boldsymbol{c}_1,\ldots,\boldsymbol{c}_n\}\) forms an orthonormal basis in \(\mathbb{R}^n.\) Define by columns the \(n\times n\) matrix

\[\begin{align*} \boldsymbol{C}=\begin{pmatrix} \boldsymbol{c}_1 & \cdots & \boldsymbol{c}_n \end{pmatrix}, \end{align*}\]

that verifies \(\boldsymbol{C}'\boldsymbol{C}=\boldsymbol{I}_n,\) where \(\boldsymbol{I}_n\) denotes the identity matrix of size \(n.\) (\(\boldsymbol{C}\) is an orthogonal matrix.)

Define the vector

\[\begin{align*} \boldsymbol{Z}=\boldsymbol{C}'\boldsymbol{X}=(\boldsymbol{c}_1'\boldsymbol{X},\ldots, \boldsymbol{c}_p'\boldsymbol{X},\ldots, \boldsymbol{c}_n'\boldsymbol{X})'=(Z_1,\ldots, Z_p,\ldots, Z_n)'. \end{align*}\]

Then:

\[\begin{align*} \mathbb{E}[\boldsymbol{Z}]=\boldsymbol{0}, \ \mathbb{V}\mathrm{ar}[\boldsymbol{Z}]=\boldsymbol{C}' \mathbb{V}\mathrm{ar}[\boldsymbol{X}] \boldsymbol{C}=\sigma^2 \boldsymbol{C}'\boldsymbol{C} =\sigma^2 \boldsymbol{I}_n. \end{align*}\]

Therefore, since \(Z_1,\ldots,Z_n\) are normal (they are linear combinations of normals) and are uncorrelated, they are independent (this is easy to see using the density version of (1.8) and (1.9)). Besides, solving in \(\boldsymbol{Z}=\boldsymbol{C}'\boldsymbol{X}\) for \(\boldsymbol{X}\) we have \(\boldsymbol{X}=\boldsymbol{C}\boldsymbol{Z}.\) Considering this and employing that \(\boldsymbol{C}'\boldsymbol{C}=\boldsymbol{I}_n,\) we get

\[\begin{align*} \boldsymbol{X}'\boldsymbol{X}=\boldsymbol{Z}\boldsymbol{C}'\boldsymbol{C} \boldsymbol{Z}=\boldsymbol{Z}'\boldsymbol{Z}. \end{align*}\]

Replacing this equality in the definition of \(Y,\) it follows that

\[\begin{align*} Y=\boldsymbol{Z}'\boldsymbol{Z}-\sum_{i=1}^p Z_i^2=\sum_{i=1}^n Z_i^2-\sum_{i=1}^p Z_i^2=\sum_{i=p+1}^n Z_i^2. \end{align*}\]

Therefore, \(Y=\sum_{i=p+1}^n Z_i^2\) is independent from \(Z_1,\ldots,Z_p.\) Also, by Corollary 2.1 it follows that

\[\begin{align*} \frac{Y}{\sigma^2}=\sum_{i=p+1}^n \frac{Z_i^2}{\sigma^2}\sim \chi_{n-p}^2. \end{align*}\]