Non-random structure

The plot below shows data from response variable \(y\) and explanatory variable \(x\). The right hand side plot shows a fitted regression line (from a simple linear regression).

Observing the residuals vs fitted values, it is clear that the residuals do not appear randomly scattered around zero. In this case, we can refer back to the original scatter plot of the data and instead of fitting a simple linear regression, for instance \(E(y_i|x_i)=\alpha + \beta x_i,\) for \(i=1\ldots,n\), then we may instead try a non-linear function of \(x\), for instance \(E(y_i|x_i)=\alpha + \beta x_i + \gamma x_i^2\). The residual plots from the latter regression model are shown below.