20.1 Conceptual Framing

20.1.1 Predictive Modeling

Predictive modeling focuses on building a function \(\hat{f}: \mathcal{X} \rightarrow \mathcal{Y}\) that maps inputs \(X\) to outputs \(Y\). For simplicity, assume:

  • \(X \in \mathbb{R}^p\) (though in practice \(X\) can be images, text, time series, etc.).
  • \(Y \in \mathbb{R}\) for regression or \(Y \in \{0, 1\}\) (or other finite set) for classification.

The yardstick for success is the function’s accuracy in out-of-sample predictions, often measured by a loss function \(L(\hat{y}, y)\). We typically choose \(\hat{f}\) to minimize expected loss:

\[ \text{(Predictive Problem)} \quad \hat{f} = \arg \min_{f \in \mathcal{F}} \mathbb{E}[L(f(X), Y)], \]

where \(\mathcal{F}\) is a class of functions (models) and \(\mathbb{E}[\cdot]\) is taken over the joint distribution of \((X, Y)\).

20.1.2 Estimation or Causal Inference

By contrast, estimation or causal inference generally aims to uncover the underlying mechanism: how does \(X\) (or a particular component \(T \subseteq X\)) cause changes in \(Y\)? The canonical problem is to estimate parameters \(\beta\) in a model \(m_\beta(x)\) such that:

\[ Y = m_\beta(X) + \varepsilon, \]

or, in linear form,

\[ Y = X\beta + \varepsilon. \]

A variety of statistical properties—consistency, unbiasedness, efficiency, confidence intervals, hypothesis tests—are relevant here. Causal interpretations usually require assumptions beyond typical i.i.d. sampling: unconfoundedness, exogeneity, or random assignment, so that \(\beta\) indeed captures how changes in \(X\) cause changes in \(Y\).

Key Distinction:

  • Prediction does not require that the parameters used in \(\hat{f}\) reflect any real-world mechanism. As long as out-of-sample predictive performance is good, the model is deemed successful—even if it’s a “black box.”
  • Causal inference demands interpretability in terms of structural or exogenous relationships. The main objective is consistent estimation of the true (or theoretically defined) parameter \(\beta\), which has an economic, biomedical, or policy interpretation.