## 12.3 Smoothing splines

These estimators use a regularized regression over the natural spline basis: placing knots at all points $$x_1,...x_n$$

For the case of cubic splines, the coefficients are the minimization of

$||y - G\beta||^2_2 + \lambda \beta^T \Omega \beta$

where $$\Omega \in R^{n \times n}$$ is the penalty matrix

$\Omega_{ij} = \int g''_i(t) g''_j(t) dt,$

and i,j = 1,..,n

and $$\lambda \beta^T \Omega \beta$$ is the regularization term used to shrink the components of $$\hat{\beta}$$ towards 0. $$\lambda > 0$$ is the tuning parameter (or smoothing parameter). Higher value of $$\lambda$$, faster shrinkage (shrinking away basis functions)

Note
smoothing splines have similar fits as kernel regression.

Smoothing splines kernel regression
tuning parameter smoothing parameter $$\lambda$$ bandwidth h