Construction of $S(\boldsymbol{\beta})$

The sum of squares for a linear model with $p$ explanatory variables is

Data: $(y_i,x_{i, 1},x_{i, 2},\ldots,x_{i, \,p}), \quad i=1,\dots,n$

Model: $E(y_{i})=\alpha+\beta_{1}x_{i, 1}+\beta_{2}x_{i, 2}+\ldots+\beta_{p}x_{i, \, p}, \quad i=1,\ldots n.$

We have defined vectors $\mathbf{Y}$ and $\boldsymbol{\beta}$ and design matrix $\mathbf{X}$ such that $$\mathrm{E}(\mathbf{Y}) = \mathbf{X}\boldsymbol{\beta},$$

Let $\mathbf{X_i}$ be the $i^{th}$ row of the $\mathbf{X}$ matrix. Therefore, $$\begin{aligned} S(\boldsymbol{\beta}) &=\sum_{i=1}^n(y_i-\mathrm{E}(y_i))^2\\ \mathrm{or}\quad S(\boldsymbol{\beta}) &=\sum_{i=1}^n(y_i-\mathbf{X_i}\boldsymbol{\beta})^{2}. \end{aligned}$$

The sum of squares function for the parameters $\boldsymbol{\beta}$ can be written in vector-matrix form as:

$$S(\boldsymbol{\beta}) = \sum_{i=1}^n(y_i-\mathrm{E}(y_i))^2 =(\mathbf{Y}-E(\mathbf{Y}))^T(\mathbf{Y}-E(\mathbf{Y}) ) = (\mathbf{Y}-\mathbf{X}\boldsymbol{\beta})^T(\mathbf{Y}-\mathbf{X}\boldsymbol{\beta} ).$$

To verify the dimensions of matrices in the vector-matrix definition of the sum of squares we need the dimensions of $S(\boldsymbol{\beta}) = (\mathbf{Y}-\mathbf{X}\boldsymbol{\beta})^T(\mathbf{Y}-\mathbf{X}\boldsymbol{\beta})$ to match on both sides of the equation and all the matrix operations to be valid.

• $\mathbf{X}$ has $n \times [p+1]$ rows and $\boldsymbol{\beta}$ has $[p+1] \times 1$.

• $\mathbf{X}\boldsymbol{\beta}$ is a valid operation and the dimension of $\mathbf{X}\boldsymbol{\beta}$ is $n \times 1$.

• $\mathbf{Y}$ is another $n \times 1$ vector.

• $\mathbf{Y}-\mathbf{X}\boldsymbol{\beta}$ is a valid operation.

• $(\mathbf{Y}-\mathbf{X}\boldsymbol{\beta})$ is $n \times 1$ and $(\mathbf{Y}-\mathbf{X}\boldsymbol{\beta})^T$ is $1\times n$.

• $(\mathbf{Y}-\mathbf{X}\boldsymbol{\beta})^T(\mathbf{Y}-\mathbf{X}\boldsymbol{\beta})$ of dimension $1 \times 1$, which is just a scalar.

• The dimension of $S(\boldsymbol{\beta})$, which is also a scalar.

Please note here, based on how the model was defined, we have $p$ explanatory variables and $p+1$ regression coefficients.