Chapter 8 Item FIt Measures

8.0.1 Learnng objectives for this chapter

In this chapter, we will explore following topics relevant to item fit measures.

  • Define item fit in the context of Cognitive Diagnostic Models (CDMs) and explain its importance for diagnostic assessment.

  • Compare and contrast absolute and relative item fit measures, and explain when each type of fit measure is most applicable.

  • Recognize and explain commonly used item fit indices.

  • Utilize diagnostic tools and statistical software to assess the fit of items in a cognitive diagnostic model, and propose solutions for dealing with misfitting items.

8.0.2 Item level fit measures

  • Item fit measures in Cognitive Diagnostic Modeling (CDM) are used to assess how well individual items in a test align with the underlying cognitive attributes or skills that the model is designed to measure.

  • Let us recall : CDMs aim to provide fine-grained information about specific skills or knowledge components required to solve each test item, rather than just an overall score.

Ensuring that each item accurately reflects these attributes is essential for drawing valid inferences about a learner’s cognitive strengths and weaknesses.

8.0.3 Types of Item-level fit Measures

Just as global/test level fit, we have two different types of item-level fit measures in CDM.

  • Absolute Fit: Absolute fit provide us with measures about how well the data from a given item align with the CDM model’s predictions. For example, comparing observed response patterns with expected ones.

  • Relative Fit: This involves comparing different models to determine which one better represents the data for a given item, usually by applying model fit statistics.

8.0.4 Implications of item-level misfit

Items that do not fit well with the model may indicate that: - Item is not well designed to measure the intended cognitive attributes - the Item may be influenced by factors not accounted for in the model (e.g., guessing, item difficulty, or test-taker fatigue).

Let us consider few situations:

  • We can apply various model for different items in CDM. First which model should we use for specific items?

  • Your test level fit statistics is showing poor fit. Do you think your data is not fitting the CDM model properly?

In both these cases, we can start exploring the item-level fit statistics. For a starter, you can start exploring whether specific items are fitting specific items properly.