10.11 Estimating Classficiation Accuracy using Multiple Imputations.

Note that when estimating classification accuracy, item paraemters are assumed to be known. In practice, item parameters can only be estimated.

• When sample size is small, item parameters may be poorly estimated and classification accuracy may be overestimated.

• Kreitchmann et al (2022) proposed to use multiple imputation to improve the estimation accuracy of clssification accuracy.

The proposed procedure can be implemented in cdmTools package. Be advised that it would be quite slow.

• Multiple imputation is a statistical technique used to account for uncertainty by generating several plausible versions of the unknown values (i.e., true mastery profiles) and averaging the results to obtain more robust estimates of classification accuracy.

Code

library(GDINA)
library(cdmTools)

## ==========================================================
## cdmTools Package [Version 1.0.5; 2024-01-04]
## More information: https://github.com/Pablo-Najera/cdmTools
## ==========================================================

Code

Y <- ecpe$dat
Q <- ecpe$Q
lcdm <- GDINA(Y, Q, model = "GDINA", verbose = 0)
CA(lcdm)

## Classification Accuracy 
## 
## Test level accuracy =  0.7522 
## 
## Pattern level accuracy: 
## 
##    000    100    010    001    110    101    011    111 
## 0.8932 0.0000 0.0000 0.4235 0.3092 0.1831 0.5902 0.9062 
## 
## Attribute level accuracy: 
## 
##     A1     A2     A3 
## 0.8978 0.8560 0.9165

Code

#CA.MI(lcdm, "MAP")