9.1 Nonlinear Mixed Models

A general form of a nonlinear mixed model is:

\[ Y_{ij} = f(\mathbf{x}_{ij}, \boldsymbol{\theta}, \boldsymbol{\alpha}_i) + \epsilon_{ij} \]

for the \(j\)-th response from the \(i\)-th cluster (or subject), where:

  • \(i = 1, \ldots, n\) (number of clusters/subjects),
  • \(j = 1, \ldots, n_i\) (number of observations per cluster),
  • \(\boldsymbol{\theta}\) represents the fixed effects,
  • \(\boldsymbol{\alpha}_i\) are the random effects for cluster \(i\),
  • \(\mathbf{x}_{ij}\) are the regressors or design variables,
  • \(f(\cdot)\) is a nonlinear mean response function,
  • \(\epsilon_{ij}\) represents the residual error, often assumed to be normally distributed with mean 0.

NLMMs are particularly useful when the relationship between predictors and the response cannot be adequately captured by a linear model.