Chapter 26 A Generalized Linear Model for Bernoulli Response Data

Example: for each \(i = 1, \ldots, n\), \(y_i \sim \text{Bernoulli}(\pi_i)\), \(\pi = \frac{\exp(x_i'\beta)}{1 + \exp(x_i'\beta)}\) and \(y_1, \ldots, y_n\) are independent. This model is called a logistic regression model.

The function \(g(\pi) = \log(\frac{\pi}{1-\pi})\) is called logit function. \(\log(\frac{\pi}{1-\pi})\) is called log(odds).

Note that \(g(\pi) = x_i'\beta\). In GLM terminology, the logit is called the link function. However, for GLMs, it is not necessary that the mean of \(y_i\) be a linear function of \(\beta\). Here are some other link functions for logistic regression:

• probit: \(\Phi^{-1}(\pi) = x'\beta\)
• complementary log-log (cloglog in R): \(\log(-\log(1-\pi)) = x'\beta\)

For GLMs, Fisher’s Scoring Method is typically used to obtain an MLE for \(\beta\), denote as \(\hat\beta\). Fisher’s Scoring Method is a variation of the Newton-Raphson algorithm in which the Hessianm atrix (matrix of second partial derivatives) is replaced by its expected value (-Fisher Information matrix).

For sufficiently large samples, \(\hat\beta\) is approximately normal with mean \(\beta\) and a variance-covariance matrix that can be approximated by the estimated inverse of the Fisher information matrix, i.e. \(\hat\beta \sim N(\beta, I^{-1}(\beta))\).

The Odds ratio: \(\frac{\tilde{\pi}}{1-\tilde{\pi}}/\frac{\pi}{1-\pi} = \exp(\beta_j)\). This can be explained as: A one unit increase in the jth explanatory variable (with all other explanatory variables held constant) is associated with a multiplicative change in the odds of success by the factor \(\exp(\beta_j)\).

If \((L_j, U_j)\) is a \(100(1-\alpha)\%\) confidence interval for \(\beta_j\). then \((\exp(L_j), \exp(U_j))\) is a \(100(1-\alpha)\%\) confidence interval for \(\exp(\beta_j)\). Also, a \(100(1-\alpha)\%\) CI for \(\pi\) is \(\left(\frac{1}{1+\exp(-L_j)},\frac{1}{1+\exp(-U_j)}\right)\).