7.8 Full information statistics- Test level (Cont’d)

Based on the full information statistics, classical goodness of fit statistics such as Pearson’s $\chi^2$ and likelihood ratio $G^2$ are widely used and fundamental fit indices in CDM analysis(Han & Johnson, 2019).

Two full-information statistics can be defined as $$\chi^2=N\sum_y \frac{(p_y-\hat{\pi}_y)^2}{\hat{\pi}_y}$$ and $$G^2=2N\sum_y p_y\frac{p_y}{\hat{\pi}_y},$$

where,

• $p_y$ is the observed proportion for response vector y.

• $\hat{\pi_y}$ is the model- based expected proportion for response vector y.

Both statistics are $\chi^2$ distributed. The associated degrees of freedom is equal to the number of response patterns - the number of parameters - 1 (i.e., $2^k-p-1$).

• $H_0$: The model fits data well
• $H_1$: The model does not fit data well

Once you perform the $\chi^2$ and $G^2$ tests, you will get a $P-value$. If the $P-value > 0.05$, we may suggest that the model fits the data well.

7.8.1 Limitations:

Both $\chi^2$ and $G^2$ suffer from the issues of Sparsity, if:

• The number of items is large, i.e., too many latent class.
• The sample size is low.
• With large item numbers, Pearson’s $\chi^2$ and $G^2$ may not follow $\chi^2$ distribution(Han & Johnson, 2019)

We can still use full information statistics in case we have large number of items, and smaller sample size by employing resampling and bootstrapping procedures. However, these process are often not easy to implement and the computation may be prohibitively expensive.

To address the issue, Maydeu-Olivares & Joe (2005) introduced an alternative approach employing limited information statistics.

References

Han, Z., & Johnson, M. S. (2019). Global-and item-level model fit indices. Handbook of Diagnostic Classification Models: Models and Model Extensions, Applications, Software Packages, 265–285.

Maydeu-Olivares, A., & Joe, H. (2005). Limited-and full-information estimation and goodness-of-fit testing in 2 n contingency tables: A unified framework. Journal of the American Statistical Association, 100(471), 1009–1020.