Chapter 10 M10: Optimization Choices

This module explores a new set of techniques for handling problems in regression (including multicollinearity and violations of some other regression conditions). Here, we change the optimization criterion: the thing we maximize or minimize in order to find the “best” coefficient estimates.

There is some textbook reading associated with this module:

• Section 6.2 “Shrinkage Methods,” bearing in mind the following notes:
  • Skim the subsection “Another Formulation for Ridge Regression and the Lasso,” starting on p. 243, and the following subsection “The Variable Selection Property of the Lasso.”
  • Skim (or skip) the subsection “A Simple Special Case for Ridge Regression and the Lasso” starting on p. 247.
  • Skip the subsection “Bayesian Interpretation for Ridge Regression and the Lasso” starting on p. 248 – unless you are interested in Bayesian analysis, in which case, enjoy.