The Aitken Model
- Orthogonal Matrices: A square matrix \(P\) is said to be orthogonal if and only if \(P'P = I\)
- Spectral Decomposition Theorem: An \(n\times n\) symmetric matrix \(H\) may be decomposed as \(H = P\Lambda P' = \sum_{i=1}^n \lambda_i p_ip_i'\)
- \(P\) is an \(n\times n\) orthogonal matrix whose columns \(p_1,\ldots, p_n\) are the orthonormal eigenvectors of \(H\)
- \(\Lambda = \text{diag}(\lambda_1, \ldots, \lambda_n)\) is a diagonal matrix whose diagonal entries are the eigenvalues of \(H\)
- Symmetric Square Root Matrix: \(H\) is an \(n\times n\) symmetric, NND matrix. Then there exists a symmetric, NND matrix \(B\) such that \(BB = H\) where \(B = P\Lambda^{1/2}P'\).
- Aitken Model: \(y = X\beta + \epsilon\), \(E(\epsilon) = 0\), \(Var(\epsilon) = \sigma^2 V\) where \(V\) is a known positive definite variance matrix.
- \(V^{-1/2}y = V^{-1/2}X\beta + V^{-1/2}\epsilon\)
- With \(z = V^{-1/2}y\), \(W = V^{-1/2}X\) and \(\delta = V^{-1/2}\epsilon\), we have \(z = W\beta + \delta\), \(E(\delta) = 0\) and \(Var(\delta) = \sigma^2I\), which is a Gauss-Markov Model.
- \(\hat z = V^{-1/2}X(X'V^{-1}X)^-X'V^{-1}y\) so that \(\hat y = X(X'V^{-1}X)^-X'V^{-1}y\).
- If \(C\beta\) is estimable, we know the BLUE is the OLS estimator. \(C(W'W)^-W'z = C(X'V^{-1}X)^-X'V^{-1}y \equiv C\hat\beta_{V}\) is called a Generalized Least Squares (GLS) estimator.
- \(\hat\beta_V = (X'V^{-1}X)^-X'V^{-1}y\) is a solution to the Aitken Equations: \(X'V^{-1}X\beta = X'V^{-1}y\). When \(V\) is diagonal, \(\hat\beta_V\) is called weighted least squares estimator.
- An unbiased estimator of \(\sigma^2\) is
\[
\frac{z'(I-P_W)z}{n - rank(W)} = \frac{\|(I- V^{-1/2}X(X'V^{-1}X)^-X'V^{-1/2})V^{-1/2}y\|^2}{n - rank(V^{-1/2}X)} = \frac{\|V^{-1/2}(y - X\hat\beta_V)\|^2}{n - r}
\]