Chapter 15 Backfitting Algorithm
Definition 15.1 Backfitting is an iterative algorithm widely used for fitting additive models and related flexible regression techniques.
Unlike classical multiple regression, which assumes a strictly linear relationship between predictors and the response, additive models allow each predictor to have its own smooth, possibly nonlinear effect.
Example:
\[ y_i = f_1(x_{1i}) + f_2(x_{2i}) + \cdots + f_p(x_{pi}) + \varepsilon_i \] where \(f_k(.)\) is a smooth function (e.g. a linear function, a quadratic function, etc…).
15.1 Backfitting Algorithm for a Linear Model
Suppose the model is
\[ \textbf{y} = \textbf{x}_1\beta_1 + \textbf{x}_2\beta_2 + \cdots + \textbf{x}_p\beta_p +\boldsymbol{\varepsilon} \] where \(\mathbb{E}(\boldsymbol{\varepsilon})=\textbf{0}\) and \(Var(\boldsymbol{\varepsilon}) = \sigma^2\textbf{I}\)