Summary points

In lecture 4, we

• Defined a linear model in vector-matrix notation \(\mathbf{Y} = \mathbf{X}\boldsymbol{\beta}+\boldsymbol{\epsilon}\).

• Construted the formula for least squares estimators \(\boldsymbol{\hat{\beta}} = (\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T\mathbf{Y}\).

• Derived the residual sum of squares \(RSS=\mathbf{Y}^T\mathbf{Y}-\mathbf{Y}^T\mathbf{X}\boldsymbol{\hat{\beta}}.\)

These formulae cover all least-squares problems, for suitably defined \(\mathbf{X}\) and \(\boldsymbol{\beta}\) they are worth remembering. To solve a particular problem, you just need to plug in the correct \(\mathbf{X}\). For any example, we simply need to identify the matrix \(\mathbf{X}\) and the vectors \(\mathbf{Y}\) and \(\boldsymbol{\beta}\) and then apply these two key results. Remember that evaluating the sum of squares at the least squares estimates for \(\boldsymbol{\beta}\) gives the residual sum of squares.