7.16 Assessing Model Accuracy
- There are various measure of model accuracy (depend on outcome)
- If outcome binary we can use the below
- Training error rate: the proportion of mistakes that are made if we apply estimate to the training observations
- 1n∑ni=1I(yi≠ˆyi): Fraction of incorrect classific
- ˆyi: predicted class label for observation i
- I(yi≠ˆyi): indicator variable that equals 1 if yi≠ˆyi and zero yi=ˆyi
- If I(yi≠ˆyi)=0 then the ith observation was classified correctly (otherwise missclassified)
- 1n∑ni=1I(yi≠ˆyi): Fraction of incorrect classific
- Test error rate: Associated with a set of test observations of the form (x0,y0)
- Ave(I(y0=ˆy0))
- ˆy0: predicted class label that results from applying the classifier to the test observation with predictor x0
- Ave(I(y0=ˆy0))
- Good classifier: One for which the test error is smallest
- The opposite of the error rate is the Correct Classification Rate (CCR)
- How many were correctly classified?
- Source: James et al. (2013, chap. 2.2.3)
References
James, Gareth, Daniela Witten, Trevor Hastie, and Robert Tibshirani. 2013. An Introduction to Statistical Learning: With Applications in R. Springer Texts in Statistics. Springer.