5.4 Generalized Least Squares

Consider

\[ \mathbf{y = X\beta + \epsilon} \]

where,

\[ var(\epsilon) = \mathbf{G} = \left( \begin{array} {cccc} g_{11} & g_{12} & ... & g_{1n} \\ g_{21} & g_{22} & ... & g_{2n} \\ . & . & . & . \\ g_{n1} & . & . & g_{nn}\\ \end{array} \right) \]

The variances are heterogeneous, and the errors are correlated.

\[ \mathbf{\hat{b}_G = (X'G^{-1}X)^{-1}X'G^{-1}Y} \]

if we know G, we can estimate \(\mathbf{b}\) just like OLS. However, we do not know G. Hence, we model the structure of G.