9.3 GAM

Generalized additive models (GAM) allow for non-linear relationships between each feature and the response by replacing each linear component \(\beta_j x_{ij}\) with a nonlinear function \(f_j(x_{ij})\). The GAM model is of the form

\[y_i = \beta_0 + \sum{f_j(x_{ij})} + \epsilon_i.\]

It is called an additive model because we calculate a separate \(f_j\) for each \(X_j\), and then add together all of their contributions.

The advantage of GAMs is that they automatically model non-linear relationships so you do not need to manually try out many different transformations on each variable individually. And because the model is additive, you can still examine the effect of each \(X_j\) on \(Y\) individually while holding all of the other variables fixed. The main limitation of GAMs is that the model is restricted to be additive, so important interactions can be missed unless you explicitly add them.