Cognitive Diagnosis Modeling
1
Introduction
2
An Introduction to Diagnostic Assessment and Modeling
2.1
Assessments of Learning
2.2
Assessments for Learning
2.3
A comparion
2.4
Educational assessments: reasoning from evidence
2.4.1
IRT as psychometric models
2.4.2
CDM as psychometric models
2.4.3
IRT vs CDM
2.5
An example: A proportional reasoning test
2.5.1
Data
2.5.2
CTT/IRT analysis
2.5.3
CDM analysis
2.6
Some Terms
2.6.1
Attribute
2.6.2
Diagnosis
2.6.3
Latent class
2.7
CDM inputs and outputs
2.7.1
Input: Item responses
2.7.2
Input: Q-matrix
2.7.3
Output: item parameter estimates
2.7.4
Output: Population-level profile estimates
2.7.5
Output: Individual-level profile estimates
3
CDMs for dichotomous data
3.1
MIRT vs CDMs
3.1.1
Multidimensional item response theory (IRT) models
3.1.2
Cognitive Diagnosis models
3.2
CDM Notations
3.3
Attribute Profiles
3.4
Condensation Rules
3.5
The DINA model
3.6
The DINO model
3.7
Reduced RUM
3.8
The LLM
3.9
The Generalized DINA model
3.10
G-DINA model: Link functions and reduced models
3.10.1
G-DINA and DINA
3.10.2
GDINA vs DINO
3.10.3
G-DINA vs R-RUM
3.10.4
G-DINA vs LLM and A-CDM
3.11
Joint Attribute Distribution
3.12
The saturated model for joint attribute distribution
3.13
The independent model for joint attribute distribution
3.14
The multivariate normal model for joint attribute distribution
3.15
The higher-order model for joint attribute distribution
3.16
The hierarchical structure model for joint attribute distribution
3.17
Assignment I
4
R Lab I: CDM Analysis
4.1
Resources for Learning the Package
4.2
Features of the GDINA R package
4.3
Data and Q-matrix
4.4
GDINA model estimation
4.5
DINA model estimation
4.6
ACDM estimation
4.7
R-RUM Estimation
4.8
Estimation of A combination of models
4.9
Estimation of a higher-order model
4.10
Estimation of CDMs with attribute hierarchy
4.11
Exercises
5
Model Estimation
5.1
Model Estimation Approaches
5.2
Likelihood of an individual response vector
5.3
Likelihood of an individual response vector (Cont’d)
5.4
Joint maximum likelihood estimation
5.5
Marginalized likelihood of an individual response vector
5.6
Marginalized maximum likelihood estimation via EM algorithm
5.7
E-step
5.8
M-step
5.9
Joint attribute distribution parameters
5.10
Quanitities in GDINA R package
5.11
Bayesian approach for parameter estimation
5.12
MCMC for DINA model in Nimble R package
5.13
MCMC for DINA model in JAGS and R2jags R package
5.14
MCMC for DINA model in STAN
6
Model Identifiability
6.1
Global Identifiability
6.1.1
What parameters we need to consider in CDMs?
6.1.2
Global identifiability in CDMs
6.2
Global identifiability of the DINA and DINO model
6.3
Global Identifiability for General CDMs
6.4
Generic Identifiability for More General CDMs
6.5
Partial Identifiability of Attribute Profiles
6.6
Partial identifiability: Completeness of the Q-matrix
6.7
Local Identifiability
6.8
Identifiability for MCMC
6.9
Assignment 2
7
Model-data Fit Evaluation
7.1
Relative Model-Data Fit at Test Level
7.1.1
Information criterions as a function of Maximum Likelihood(ML) of CDM models
(Chen et al., 2013)
7.2
Relative Model-Data Fit at Test Level (Cont’d)
7.3
Relative Model-Data Fit at Test Level (Cont’d)
7.3.1
Calculating number of model paraters for CDMs
7.4
Relative Model-Data Fit at Test Level (Cont’d)
7.5
Relative Model-Data Fit at Test Level : Practice Session in R
7.6
Test-level Absolute Fit measures
7.7
Full information statistics
7.8
Full information statistics- Test level (Cont’d)
7.8.1
Limitations:
7.9
Limited information statistics
7.10
Limited information statistics (Cont’d)
7.11
Limited information statistics (Cont’d)
7.12
Standardized Root Mean Squared Residual (SRMSR)
7.13
Practice Session: Calculating Absolute Fit Measures
7.14
Absolute Fit Measures in the Bayesian Approach of CDM estimation
7.14.1
Posterior Predictive Model Checking
7.15
Relative Fit Measures in the Bayesian Approach of CDM estimation
7.15.1
Deviance Information Criterion(DIC)
7.15.2
Watanabe-Akaike Information Criterion(WAIC)
7.16
Exercises
8
Item FIt Measures
8.0.1
Learnng objectives for this chapter
8.0.2
Item level fit measures
8.0.3
Types of Item-level fit Measures
8.0.4
Implications of item-level misfit
8.1
Item-level Absolute Fit measures
8.1.1
Performing hypothesis tests:
8.2
Item-level Absolute Fit measures (Cont’d)
8.3
Item-level Relative Fit measures
8.4
Item-level Relative Fit measures (Cont’d)
8.5
Item-level Relative Fit measures (Cont’d)
8.6
Item-level Relative Fit measures (Cont’d)
8.7
Item-level Relative Fit measures (Cont’d)
8.8
Item-level Relative Fit measures (Cont’d)
8.9
Item-level Relative Fit measures (Cont’d)
8.10
Item-level Relative Fit measures (Cont’d)
8.11
Item-level Relative Fit measures (Cont’d)
8.12
Exercise 1
9
ATTRIBUTE PROFILE ESTIMATION
9.1
Maximum Likelihood Estimation (MLE)
9.2
Maximum a Posterior (MAP) Estimation
9.3
Expected a Posterior (EAP) Estimation
9.4
Attribute Estimation in GDINA R package
9.5
Exercises
References
Published with bookdown
Handout for Cognitive Diagnosis Modeling
2.3
A comparion
