## 12.9 Random forests

• Random Forests (RFs) provide improvement over bagged trees
• Decorrelated trees lead to reduction in both test and OOB error over bagging
• RFs also build decision trees on bootstrapped training samples but…
• Each time a split in a tree is considered,a random sample of $$m$$ predictors is chosen as split candidates from the full set of $$p$$ predictors
• Split only allowed to use one of $$m$$ predictors
• Fresh sample of $$m$$ predictors taken at each split (typically not all but $$m \approx \sqrt{p}$$!)
• On average $$(p-m)/p$$ splits won’t consider strong predictor (decorrelating trees)
• Main difference bagging vs. random forests: Choice of predictor subset size $$m$$
• RF built using $$m=p$$ equates Bagging
• Recommmendation: Vary $$m$$ and use small $$m$$ when predictors are highly correlated