12.2 Introduction (Cont’d)

• There are a large number of approaches to create a Q-matrix based on the data or refine a Q-matrix developed by domain experts.

• This session focuses on empirically-based methods of validating a Q-matrix developed by domain experts introduced in the following articles

  • PVAF method (de la Torre & Chiu, 2016)
  • Modified PVAF method (Nájera et al., 2019)
  • Mesa plot (de la Torre & Ma, 2016)
  • Stepwise method (Ma & de la Torre, 2020)

• These methods are based on the G-DINA model so applicable to all the reduced models it subsumes

  • One should fit the G-DINA model to the data to avoid the potential misspecification of models
  • When true model is a reduced model subsumed by the G-DINA, these procedures will also work

• These methods are based solely on the data at hand, and are intended to complement, not replace, other methods of checking model fit

  • One needs the data (item responses) and a largely correct provisional Q-matrix

• A more integrative framework of validating a Q-matrix should include other sources of information (e.g., expert opinion)

  • The procedures introduced here are purely data-driven
  • experts’ option may need to be taken account

References

de la Torre, J., & Chiu, C.-Y. (2016). A general method of empirical q-matrix validation. Psychometrika, 81(2), 253–273. https://doi.org/10.1007/s11336-015-9467-8

de la Torre, J., & Ma, W. (2016). Cognitive diagnosis modeling: A general framework approach and its implementation in r.

Ma, W., & de la Torre, J. (2020). An empirical q-matrix validation method for the sequential generalized DINA model. The British Journal of Mathematical and Statistical Psychology, 73(1), 142–163. https://doi.org/10.1111/bmsp.12156

Nájera, P., Sorrel, M. A., & Abad, F. J. (2019). Reconsidering cutoff points in the general method of empirical q-matrix validation. Educational and Psychological Measurement, 79(4), 727–753. https://doi.org/10.1177/0013164418822700