9.12 Trade-Off: Prediction Accuracy vs. Model Interpretability

• Some ML methods are less flexible (shape of f), e.g., linear regression can only generate linear functions (Example: Prediction: Linear model (Visualization))
• James et al. (2013, 25), Fig. 2.7. provides an overview
• Q: Why would we ever choose to use a more restrictive method instead of a very flexible approach?
• Q: Is high flexibility always desirable?
• Debate around interpretable machine learning
  • Interpretable Machine Learning: A Guide for Making Black Box Models Explainable

• Low flexibility: Better interpretation if aim is inference (association between predictor and outcome) BUT potentially worse prediction
• High flexibility: Can also yield worse predictions because of overfitting (counterintuitive!)

References

James, Gareth, Daniela Witten, Trevor Hastie, and Robert Tibshirani. 2013. An Introduction to Statistical Learning: With Applications in R. Springer Texts in Statistics. Springer.