Chapter 10 M10: Optimization Choices
This module explores a new set of techniques for handling problems in regression (including multicollinearity and violations of some other regression conditions). Here, we change the optimization criterion: the thing we maximize or minimize in order to find the “best” coefficient estimates.
There is some textbook reading associated with this module:
- Section 6.2 “Shrinkage Methods,” bearing in mind the following notes:
- Skim the subsection “Another Formulation for Ridge Regression and the Lasso,” starting on p. 243, and the following subsection “The Variable Selection Property of the Lasso.”
- Skim (or skip) the subsection “A Simple Special Case for Ridge Regression and the Lasso” starting on p. 247.
- Skip the subsection “Bayesian Interpretation for Ridge Regression and the Lasso” starting on p. 248 – unless you are interested in Bayesian analysis, in which case, enjoy.