6.34 Trade-Off: Prediction Accuracy vs. Model Interpretability

• Some ML methods are less flexible (shape of f), e.g., linear regression can only generate linear functions (Example: Prediction: Linear model (Visualization))
• Fig. 2.7. (James et al. 2013, 25) provides an overview
• Q: Why would we ever choose to use a more restrictive method instead of a very flexible approach?
• Q: Is high flexibility always desirable?

• Low flexibility: Better interpretation if aim is inference (association between predictor and outcome) BUT potentially worse prediction
• High flexibility: Can also yield worse predictions because of overfitting (counterintuitive!)

References

James, Gareth, Daniela Witten, Trevor Hastie, and Robert Tibshirani. 2013. An Introduction to Statistical Learning: With Applications in R. Springer Texts in Statistics. Springer.