7.9 Limited information statistics

Limited-information statistics focus on marginal proportions or probabilities. The primary idea of the limited information statistics is to utilize r-th order moments ($\pi_r$) instead of using proportions of all possible response patterns.

M. Hansen et al. (2014) explored the utility of limited information matrix for CDMs, and suggested that $M_2$ is more consistent in detecting test-level model misspecification. Hence, let us explore about $M_2$ statistic to assess the test lelvel model data fit.

We consider $M_2$ statistic, which only focuses on the first two marginals.

The first marginal proportions are the proportions of students who answer each item correctly:

$$\mathbf{p}_1=\big[P(Y_{1}=1),\ldots,P(Y_{j}=1),\ldots,P(Y_{J}=1)\big]$$

The second marginal proportions are the proportions of students who answer a pair of items correctly:

$$\mathbf{p}_2=\big[P(Y_{1}=1,Y_{2}=1),\ldots,P(Y_{j}=1,Y_{j'}=1),\ldots,P(Y_{J-1}=1,Y_{J}=1)\big]$$ Let us try to understand the values of $\mathbf{p}_1$ and $\mathbf{p}_2$

Code

df<-data1[1:10,1:5]
df

##    Item1 Item2 Item3 Item4 Item5
## 1      1     1     1     1     1
## 2      1     1     1     1     0
## 3      1     0     1     0     1
## 4      1     0     1     0     1
## 5      1     0     0     0     0
## 6      1     0     1     1     1
## 7      1     1     0     1     0
## 8      1     1     1     1     1
## 9      1     1     0     1     0
## 10     1     1     1     1     0

1st order probabilities,

• $P(Y_{1}=1) = 10/10$
• $P(Y_{2}=1) = 6/10$ and so on.

Second order probabilities,

• $P(Y_{1}=1,Y_{2}=1)= 6/10=0.6$
• $P(Y_{2}=1,Y_{3}=1)= 4/10=0.4$

and so on.

We can also find the model-predicted counterparts (i.e., $\hat{\pi_1}$ and $\hat{\pi_2}$) in similar way.

References

Hansen, M., Cai, L., Monroe, S., & Li, Z. (2014). Limited-information goodness-of-fit testing of diagnostic classification item response theory models. CRESST report 840. National Center for Research on Evaluation, Standards, and Student Testing (CRESST).