6.4 Generic Identifiability for More General CDMs

Sufficient and necessary global identifiability conditions are difficult to find for R-RUM, ACDM, LLM and G-DINA model. Researchers have considered a generic identifiability, which ensures that the model is identified for all most all parameter space.

Xu (2019) points out that main-effect models and saturated models are generically identified when the Q-matrix takes the following form

\[ Q=\begin{pmatrix} Q_1 \\ Q_2\\ Q^{\star} \end{pmatrix} \]

where \(Q_1\) and \(Q_2\) take the following forms (\(*\) can be either 0 or 1):

\[ Q_i=\begin{pmatrix} 1 & * &\ldots&*\\ * & 1 &\ldots&*\\ \vdots & \vdots &&\vdots\\ *&*&\ldots&1 \end{pmatrix} \] and in \(Q^{\star}\), each attribute is required by at least one item. In this case, the main-effects CDMs, such as R-RUM and ACDM, and the all-iteraction-effect CDMs, such as LCDM, GDINA, and GDM are generally identifiable(Xu, 2019).

Now let’s check how nonidentifiability affects LLM analysis.

Based on the generic identifiability conditions above, the Q-matrix below is not identified.

Code

Q <- matrix(c(1,0,0,
              0,1,0,
              0,0,1,
              1,0,1,
              1,1,0),ncol = 3,byrow = TRUE)

	Attribute 1	Attribute 2	Attribute 3
Item 1	1	0	0
Item 2	0	1	0
Item 3	0	0	1
Item 4	1	0	1
Item 5	1	1	0

The data in ACDM_idf.csv was simulated based on the following parameters:

Code

delta <- list(item1=c(0.2,0.5),
              item2=c(0.3,0.6),
              item3=c(0.1,0.4),
              item4=c(0.2,0.3,0.4),
              item5=c(0.2,0.2,0.5))
p <- rep(1/8,8)

Let us fit the ACDM model to the data using the code below. Please change the randomseed used to generate initial item parameters and see how that will affect the estimates.

Code

library(GDINA)
df <- read.csv("data/ACDM_idf.csv")
ACDM.est <- GDINA(df,Q,model = "ACDM",
                  control = list(conv.crit=1e-6,
                                 randomseed=12303),
                  verbose = FALSE)
coef(ACDM.est,"delta")

## $`Item 1`
##   d0   d1 
## 0.06 0.70 
## 
## $`Item 2`
##    d0    d1 
## 0.029 0.726 
## 
## $`Item 3`
##     d0     d1 
## 0.0001 0.7094 
## 
## $`Item 4`
##   d0   d1   d2 
## 0.31 0.26 0.24 
## 
## $`Item 5`
##    d0    d1    d2 
## 0.086 0.088 0.528

Code

coef(ACDM.est,"lambda")

## p(000) p(100) p(010) p(001) p(110) p(101) p(011) p(111) 
##  0.083  0.048  0.164  0.029  0.300  0.048  0.148  0.180

References

Xu, G. (2019). Identifiability and Cognitive Diagnosis Models (M. von Davier & Y.-S. Lee, Eds.; pp. 333–357). Springer International Publishing. http://link.springer.com/10.1007/978-3-030-05584-4_16