Chapter 6 Non-linear Regression

Definition: models in which the derivatives of the mean function with respect to the parameters depend on one or more of the parameters.

To approximate data, we can approximate the function

  • by a high-order polynomial
  • by a linear model (e.g., a Taylor expansion around X’s)
  • a collection of locally linear models or basis function

but it would not easy to interpret, or not enough data, or can’t interpret them globally.

intrinsically nonlinear models:

\[ Y_i = f(\mathbf{x_i;\theta}) + \epsilon_i \]

where \(f(\mathbf{x_i;\theta})\) is a nonlinear function relating \(E(Y_i)\) to the independent variables \(x_i\)

  • \(\mathbf{x}_i\) is a k x 1 vector of independent variables (fixed).
  • \(\mathbf{\theta}\) is a p x 1 vector of parameters.
  • \(\epsilon_i\)s are iid variables mean 0 and variance \(\sigma^2\). (sometimes it’s normal).