What is a linear model?
We discussed the structure of a model and identified two components, the deterministic and the stochastic (or random) parts. We saw that the deterministic part involves the predictor or explanatory variable.
In each of the examples we looked at, the model can be described as a regression model, with \(y\) regressed on \(x\). We identified the parameters, using Greek symbols (\(\alpha\), \(\beta\), \(\gamma\) etc). These are unknown quantities which must be estimated, consequently allowing us to estimate the deterministic part of the relationship, also called the regression function.
In every model we have discussed, the parameters (\(\alpha\), \(\beta\), \(\gamma\) etc) appear linearly in the deterministic component, so they are called linear models. Note this does not necessarily imply a straight line.
Statistically, a linear model is one in which the parameters appear linearly in the deterministic part \(f(x)\) of the model. For example, the quadratic regression model in the potatoes example is a linear model.
This definition is somewhat counter intuitive. As a reminder
We want the unknown parameters to appear linearly in the deterministic part \(f(x)\) of the model.
This means that our variable \(x\) doesn’t need to appear linearly
Functions that are examples of linear models
Below are some examples of functions. As a general rule, we use Greek letters for unknow parameters and we use \(x\) for variables.
\[\begin{aligned} &Y=\beta_0+\beta_1 x + \epsilon \\ &Y =\beta_0 +\beta_1 x+\beta_2 x^2 +\epsilon\\ &Y =\beta_0+\beta x_1 + \gamma x^2 +\delta \exp(5x) +\epsilon \end{aligned}\]
These are all examples of linear models