7.16 Assessing Model Accuracy

  • There are various measure of model accuracy (depend on outcome)
  • If outcome binary we can use the below
  • Training error rate: the proportion of mistakes that are made if we apply estimate to the training observations
    • 1nni=1I(yiˆyi): Fraction of incorrect classific
      • ˆyi: predicted class label for observation i
      • I(yiˆyi): indicator variable that equals 1 if yiˆyi and zero yi=ˆyi
      • If I(yiˆyi)=0 then the ith observation was classified correctly (otherwise missclassified)
  • Test error rate: Associated with a set of test observations of the form (x0,y0)
    • Ave(I(y0=ˆy0))
      • ˆy0: predicted class label that results from applying the classifier to the test observation with predictor x0
  • Good classifier: One for which the test error is smallest
  • The opposite of the error rate is the Correct Classification Rate (CCR)
    • How many were correctly classified?
  • Source: James et al. (2013, chap. 2.2.3)

References

James, Gareth, Daniela Witten, Trevor Hastie, and Robert Tibshirani. 2013. An Introduction to Statistical Learning: With Applications in R. Springer Texts in Statistics. Springer.