6.34 Trade-Off: Prediction Accuracy vs. Model Interpretability

  • Some ML methods are less flexible (shape of f), e.g., linear regression can only generate linear functions (Example: Prediction: Linear model (Visualization))
  • Fig. 2.7. (James et al. 2013, 25) provides an overview
  • Q: Why would we ever choose to use a more restrictive method instead of a very flexible approach?
  • Q: Is high flexibility always desirable?

  • Low flexibility: Better interpretation if aim is inference (association between predictor and outcome) BUT potentially worse prediction
  • High flexibility: Can also yield worse predictions because of overfitting (counterintuitive!)


James, Gareth, Daniela Witten, Trevor Hastie, and Robert Tibshirani. 2013. An Introduction to Statistical Learning: With Applications in R. Springer Texts in Statistics. Springer.