12.1 Regression Splines

To estimate the regression function \(r(X) = E(Y|X =x)\), we can fit a k-th order spline with knots at some prespecified locations \(t_1,...,t_m\)

Regression splines are functions of

\[ \sum_{j=1}^{m+k+1} \beta_jg_j \]

where

\(\beta_1,..\beta_{m+k+1}\) are coefficients \(g_1,...,g_{m+k+1}\) are the truncated power basis functions for k-th order splines over the knots \(t_1,...,t_m\)

To estimate the coefficients

\[ \sum_{i=1}^{n} (y_i - \sum_{j=1}^{m} \beta_j g_j (x_i))^2 \]

then regression spline is

\[ \hat{r}(x) = \sum_{j=1}^{m+k+1} \hat{\beta}_j g_j (x) \]

If we define the basis matrix \(G \in R^{n \times (m+k+1)}\) by \[ G_{ij} = g_j(x_i) \] where \(i = 1,..,n\) , \(j = 1,..,m+k+1\)

Then,

\[ \sum_{i=1}^{n} (y_i - \sum_{j=1}^{m} \beta_j g_j (x_i))^2 = ||y - G \beta||_2^2 \]

and the regression spline estimate at x is

\[ \hat{r} (x) = g(x)^T \hat{\beta}= g(x)^T(G^TG)^{-1}G^Ty \]