10.2 Classification Consistency
Classification consistency is the probability that two parallel forms of the assessment result in the same estimated classification.
The subsequent discussions are summarized from Cui et al. (2012).
Classification consistency often is referred to as the reliability of classifications.
The definition of classification consistency relies on data from parallel/ repeated assessments. However, it is often not easy to obtain such data.
Classification consistency is the agreement between the classification estimated from two parallel forms.
Both accuracy and consistency can be viewed as measures of agreement, either between the true attributes and estimated attributes or between attributes estimated from two parallel forms. We mainly focus on classification accuracy because it provides direct evidence about the quality of person classifications.
Classification accuracy can be defined at individual attribute level or attribute profile level. We can employ two different approaches to obtain classification accuracy: (1) The Monte Carlo Approach, and (2) Analytical Approach.