20.1 Conceptual Framing

20.1.1 Predictive Modeling

Predictive modeling focuses on building a function $\hat{f}: \mathcal{X} \rightarrow \mathcal{Y}$ that maps inputs $X$ to outputs $Y$. For simplicity, assume:

• $X \in \mathbb{R}^p$ (though in practice $X$ can be images, text, time series, etc.).
• $Y \in \mathbb{R}$ for regression or $Y \in \{0, 1\}$ (or other finite set) for classification.

The yardstick for success is the function’s accuracy in out-of-sample predictions, often measured by a loss function $L(\hat{y}, y)$. We typically choose $\hat{f}$ to minimize expected loss:

$$\text{(Predictive Problem)} \quad \hat{f} = \arg \min_{f \in \mathcal{F}} \mathbb{E}[L(f(X), Y)],$$

where $\mathcal{F}$ is a class of functions (models) and $\mathbb{E}[\cdot]$ is taken over the joint distribution of $(X, Y)$.

20.1.2 Estimation or Causal Inference

By contrast, estimation or causal inference generally aims to uncover the underlying mechanism: how does $X$ (or a particular component $T \subseteq X$) cause changes in $Y$? The canonical problem is to estimate parameters $\beta$ in a model $m_\beta(x)$ such that:

$$Y = m_\beta(X) + \varepsilon,$$

or, in linear form,

$$Y = X\beta + \varepsilon.$$

A variety of statistical properties—consistency, unbiasedness, efficiency, confidence intervals, hypothesis tests—are relevant here. Causal interpretations usually require assumptions beyond typical i.i.d. sampling: unconfoundedness, exogeneity, or random assignment, so that $\beta$ indeed captures how changes in $X$ cause changes in $Y$.

Key Distinction:

• Prediction does not require that the parameters used in $\hat{f}$ reflect any real-world mechanism. As long as out-of-sample predictive performance is good, the model is deemed successful—even if it’s a “black box.”
• Causal inference demands interpretability in terms of structural or exogenous relationships. The main objective is consistent estimation of the true (or theoretically defined) parameter $\beta$, which has an economic, biomedical, or policy interpretation.

---