15.4 Summary Table

Method Type Approach Key Criterion Notes
Mallows’s C Statistic Information Criterion Subset Selection Model Complexity vs Fit Balances fit and simplicity.
Akaike Information Criterion (AIC) Information Criterion Model Selection Minimizes AIC Penalizes model complexity.
Bayesian Information Criterion (BIC) Information Criterion Model Selection Minimizes BIC Stronger penalty for complexity.
Hannan-Quinn Criterion (HQC) Information Criterion Model Selection Minimizes HQC Combines AIC and BIC features.
Minimum Description Length (MDL) Information Criterion Model Selection Data + Model Encoding Costs Focuses on encoding efficiency.
Prediction Error Sum of Squares (PRESS) Error-Based Cross-Validation Minimizes Prediction Error Measures predictive accuracy.
Best Subsets Algorithm Exhaustive Search Subset Selection Best Fit Across Subsets Considers all variable combinations.
Forward Selection Stepwise Add Variables Significance Testing Adds variables one at a time.
Backward Elimination Stepwise Remove Variables Significance Testing Removes variables iteratively.
Stepwise (Both Directions) Stepwise Add/Remove Variables Significance Testing Combines forward and backward methods.
Branch-and-Bound Algorithm Optimized Search Subset Selection Efficient Subset Search Avoids exhaustive search.
Recursive Feature Elimination (RFE) Iterative Optimization Feature Removal Model Performance Removes least important predictors.
Genetic Algorithms Heuristic Search Evolutionary Process Fitness Function Mimics natural selection for subsets.