## 21.2 Classifier performance & fairness (2)

• Algorithmic fairness can be assessed with respect to an input characteristic $$C$$ (e.g., race, sex)
• False positive parity…
• …with respect to characteristic C is satisfied if the false positive rate for inputs with $$C = 0$$ (e.g., black) is the same as the false positive rate for inputs with $$C = 1$$ (e.g., white)
• ProPublica found that the false positive rate for African-American defendants (i.e., the percentage of innocent African-American defendants classified as likely to re-offend) was higher than for white defendants (NO false positive parity)
• Calibration…
• …with respect to characteristic $$C$$ is satisfied if an individual who was labeled “positive” has the same probability of actually being positive, regardless of the value of $$C$$
• …and if an individual who was labeled “negative” has the same probability of actually being negative regardless of the value of $$C$$
• COMPAS makers claim that COMPAS satisfies calibration!