## 12.1 Regression Splines

To estimate the regression function $$r(X) = E(Y|X =x)$$, we can fit a k-th order spline with knots at some prespecified locations $$t_1,...,t_m$$

Regression splines are functions of

$\sum_{j=1}^{m+k+1} \beta_jg_j$

where

$$\beta_1,..\beta_{m+k+1}$$ are coefficients $$g_1,...,g_{m+k+1}$$ are the truncated power basis functions for k-th order splines over the knots $$t_1,...,t_m$$

To estimate the coefficients

$\sum_{i=1}^{n} (y_i - \sum_{j=1}^{m} \beta_j g_j (x_i))^2$

then regression spline is

$\hat{r}(x) = \sum_{j=1}^{m+k+1} \hat{\beta}_j g_j (x)$

If we define the basis matrix $$G \in R^{n \times (m+k+1)}$$ by $G_{ij} = g_j(x_i)$ where $$i = 1,..,n$$ , $$j = 1,..,m+k+1$$

Then,

$\sum_{i=1}^{n} (y_i - \sum_{j=1}^{m} \beta_j g_j (x_i))^2 = ||y - G \beta||_2^2$

and the regression spline estimate at x is

$\hat{r} (x) = g(x)^T \hat{\beta}= g(x)^T(G^TG)^{-1}G^Ty$