16.3 Causation versus Prediction

Understanding the relationship between causation and prediction is crucial in statistical modeling.

Let \(Y\) be an outcome variable dependent on \(X\), and our aim is to manipulate \(X\) to maximize a payoff function \(\pi(X, Y)\) (Kleinberg et al. 2015). The decision on \(X\) hinges on:

\[ \begin{aligned} \frac{d\pi(X, Y)}{d X} &= \frac{\partial \pi}{\partial X} (Y) + \frac{\partial \pi}{\partial Y} \frac{\partial Y}{\partial X} \\ &= \frac{\partial \pi}{\partial X} \text{(Prediction)} + \frac{\partial \pi}{\partial Y} \text{(Causation)} \end{aligned} \]

Empirical work is essential for estimating the derivatives in this equation:

  • \(\frac{\partial Y}{\partial X}\) is required for causal inference to determine \(X\)’s effect on \(Y\),

  • \(\frac{\partial \pi}{\partial X}\) is required for prediction of \(Y\).

(SICSS 2018 - Sendhil Mullainathan’s presentation slide)
(SICSS 2018 - Sendhil Mullainathan’s presentation slide)


Kleinberg, Jon, Jens Ludwig, Sendhil Mullainathan, and Ziad Obermeyer. 2015. “Prediction Policy Problems.” American Economic Review 105 (5): 491–95.