10.7 $2\times 2$ Contingency Table in Practice

Let us focus on attribute $k$. For an element $p_{ab}$ in the table,

$$\begin{aligned} p_{ab}&=P(\alpha_k=a,\hat{\alpha}_k(\mathbf{Y})=b)\\ &=P(\alpha_k=a,\hat{\alpha}_k(\mathbf{Y})=b|\mathbf{Y})P(\mathbf{Y}) \end{aligned}$$ Given a finite sample $\mathbf{y}$ of size $n$, we can estimate

$$\begin{aligned} \hat{p}_{ab}&=P(\alpha_k=a,\hat{\alpha}_k(\mathbf{Y}=\mathbf{y})=b)\\ &=P(\alpha_k=a,\hat{\alpha}_k(\mathbf{Y}=\mathbf{y})=b|\mathbf{Y}=\mathbf{y})P(\mathbf{Y}=\mathbf{y})\\ &=\frac{1}{n}\sum_{i=1}^n\underbrace{P(\alpha_{ik}=a|\mathbf{y}_i)}_{\text{estimated probability of mastery/non-mastery}}\underbrace{I(\hat{\alpha}_{ik}=b|\mathbf{y}_i)}_{\text{estimated mastery status}}\\ \end{aligned}$$ in this case, we are calclluating the joint probability that the true mastery status of attribute $k$ is $a$ (either 1 for mastery or 0 for non-mastery) and the model estimated mastery as $b$ (either 0 or 1). $P(\alpha_k=a|Y=y)$ is the probability of true mastery or non-mastery for attribute $k$ given the response pattern $Y=y$. $P(\hat{\alpha}_k(Y=y)=b|Y=y)$ is the probability that the model’s estimate of mastery is $b$ given the response pattern $Y=y$.

Lest us focus on two probabilities:

• $P\left(\alpha_{i k}=a \mid \mathbf{y}_i\right)$ is the posterior probability that individual $i$ has mastery(or non-mastery) of attribute $k$, given their response vector $y_i$.

• $I\left(\hat{\alpha}_{i k}=b \mid \mathbf{y}_i\right)$ is an indicator function that is 1 if the model estimates the mastery status $b$ correctly, and 0 otherwise.