Chapter 15 Backfitting Algorithm
This chapter is still under development. If I can’t finish this, we will not discuss this anymore.
Definition 15.1 Backfitting is an iterative algorithm widely used for fitting additive models and related flexible regression techniques.
Unlike classical multiple regression, which assumes a strictly linear relationship between predictors and the response, additive models allow each predictor to have its own smooth, possibly nonlinear effect.
Example:
\[ y_i = f_1(x_{1i}) + f_2(x_{2i}) + \cdots + f_p(x_{pi}) + \varepsilon_i \] where \(f_k(.)\) is a smooth function (e.g. a linear function, a quadratic function, etc…).
15.1 Backfitting Algorithm for a Linear Model
Suppose the model is
\[ \textbf{y} = \textbf{x}_1\beta_1 + \textbf{x}_2\beta_2 + \cdots + \textbf{x}_p\beta_p +\boldsymbol{\varepsilon} \] where \(\mathbb{E}(\boldsymbol{\varepsilon})=\textbf{0}\) and \(Var(\boldsymbol{\varepsilon}) = \sigma^2\textbf{I}\)