9.1 Attribute Profile Estimation

Attribute Profile Estimation in Cognitive Diagnostic Modeling (CDM) refers to the scientific process of estimating an individual’s skills/attributes across a set of predefined attributes or skills based on their performance on assessment items. The attribute estimation process aims to provide mastery of skills/attributes in specific cognitive areas, which can inform tailored instructional strategies and interventions.

Recall: CDM is often mentioned as assessment for learning

To estimate attributes for each individual test-taker, we can employ different estimation approaches.

Kreitchmann et al. (2023) summarized the attribute profile estimation in several steps:

For Maximum Likelihood Estimation:

• We first obtain the marginal maximum likelihood of the model parameters i.e., $\hat{\delta}$ and $\hat{P(\alpha_l)}$.

• We obtain the MLE of $\hat{\alpha}$ for respondent $i$ will be of class $l$ for which likilihood for $(Y_i|\alpha_l, \delta)$ is maximum, assumming that MMLE estimates of $\delta$ is correct.

For Bayesian Estimation:

• In Bayesian approach, the estimated classification of attributes for each test-taker is estimated based on the expected or maximum posterior probability of each attribute profile known as expected-a-posterior(EAP) or maximum-a-posterior (MAP).

• Expected-A-Posterior (EAP) represents the expected value of the posterior distribution for an attribute profile.

• Maximum-A-Posterior (MAP) refers to the profile with the highest posterior probability for each examinee.

The posterior probability of attribute profile αl for examinee i is numerically approximated as

$$P(\alpha_l \mid \mathbf{Y}_i, \delta) =\frac{\operatorname{lik}\left(\mathbf{Y}_i \mid \alpha_l, \delta\right) P\left(\alpha_l\right)}{\sum_{l=1}^L \operatorname{lik}\left(\mathbf{Y}_i \mid \alpha_l, \delta\right) P\left(\alpha_l\right)}$$ - the MAP estimator classifies each examinee to its most probable attribute profile.

• The EAP estimator calculates the marginal probability for each attribute individually, then assigns each attribute to either mastery or non-mastery by comparing the probability against a set threshold (e.g., 0.5).

References

Kreitchmann, R. S., Torre, J. de la, Sorrel, M. A., Nájera, P., & Abad, F. J. (2023). Improving reliability estimation in cognitive diagnosis modeling. Behavior Research Methods, 55(7), 3446–3460.