# Chapter 6 Non-linear Regression

Definition: models in which the derivatives of the mean function with respect to the parameters depend on one or more of the parameters.

To approximate data, we can approximate the function

• by a high-order polynomial
• by a linear model (e.g., a Taylor expansion around X’s)
• a collection of locally linear models or basis function

but it would not easy to interpret, or not enough data, or can’t interpret them globally.

intrinsically nonlinear models:

$Y_i = f(\mathbf{x_i;\theta}) + \epsilon_i$

where $$f(\mathbf{x_i;\theta})$$ is a nonlinear function relating $$E(Y_i)$$ to the independent variables $$x_i$$

• $$\mathbf{x}_i$$ is a k x 1 vector of independent variables (fixed).
• $$\mathbf{\theta}$$ is a p x 1 vector of parameters.
• $$\epsilon_i$$s are iid variables mean 0 and variance $$\sigma^2$$. (sometimes it’s normal).