20.1 Conceptual Framing

20.1.1 Predictive Modeling

Predictive modeling focuses on building a function $\hat{f}: \mathcal{X} \rightarrow \mathcal{Y}$ that maps inputs $X$ to outputs $Y$ . For simplicity, assume:

$X \in \mathbb{R}^p$ (though in practice $X$ can be images, text, time series, etc.).
$Y \in \mathbb{R}$ for regression or $Y \in \{0, 1\}$ (or other finite set) for classification.

The yardstick for success is the function’s accuracy in out-of-sample predictions, often measured by a loss function $L(\hat{y}, y)$ . We typically choose $\hat{f}$ to minimize expected loss:

$\text{(Predictive Problem)} \quad \hat{f} = \arg \min_{f \in \mathcal{F}} \mathbb{E}[L(f(X), Y)],$

where $\mathcal{F}$ is a class of functions (models) and $\mathbb{E}[\cdot]$ is taken over the joint distribution of $(X, Y)$ .

20.1.2 Estimation or Causal Inference

By contrast, estimation or causal inference generally aims to uncover the underlying mechanism: how does $X$ (or a particular component $T \subseteq X$ ) cause changes in $Y$ ? The canonical problem is to estimate parameters $\beta$ in a model $m_\beta(x)$ such that:

$Y = m_\beta(X) + \varepsilon,$

or, in linear form,

$Y = X\beta + \varepsilon.$

A variety of statistical properties—consistency, unbiasedness, efficiency, confidence intervals, hypothesis tests—are relevant here. Causal interpretations usually require assumptions beyond typical i.i.d. sampling: unconfoundedness, exogeneity, or random assignment, so that $\beta$ indeed captures how changes in $X$ cause changes in $Y$ .

Key Distinction:

Prediction does not require that the parameters used in $\hat{f}$ reflect any real-world mechanism. As long as out-of-sample predictive performance is good, the model is deemed successful—even if it’s a “black box.”
Causal inference demands interpretability in terms of structural or exogenous relationships. The main objective is consistent estimation of the true (or theoretically defined) parameter $\beta$ , which has an economic, biomedical, or policy interpretation.