27.11 Evaluation of a Regression Discontinuity Design

After estimating an RD model, it is crucial to evaluate whether the assumptions hold and whether the results are robust to different specifications. The key aspects of RD evaluation include:

1. Graphical and formal evidence for a discontinuity in treatment and outcome variables.
2. Validation of RD assumptions, including:
   • The absence of discontinuities in pre-treatment covariates.
   • No manipulation of the assignment variable.
3. Robustness checks for functional form and bandwidth choice.
4. External validity: assessing whether results generalize beyond the cutoff.

A well-implemented RD should demonstrate a clear treatment effect at the cutoff while ensuring that no other discontinuous changes confound the effect.

27.11.1 Graphical and Formal Evidence

27.11.1.1 Visual Inspection of the RD Effect

A fundamental step in RD analysis is to plot the outcome variable against the running variable:

• Compute binned averages of $Y_i$ for small intervals of $X_i$.
• Overlay a smoothed polynomial regression separately for $X < c$ and $X \geq c$.
• A visible jump at $X = c$ provides initial evidence of a treatment effect.

Additionally, plotting treatment probability $P(D = 1 | X)$ ensures that assignment follows the expected RD rule.

27.11.1.2 No Discontinuity in Pre-Treatment Covariates

To rule out omitted variable bias, we check whether other covariates (age, education, prior test scores, etc.) exhibit jumps at the cutoff.

For each covariate $W_i$, estimate:

$$W_i = \alpha + f(X_i) \beta + \tau D_i + \epsilon_i.$$

• If $\tau$ is statistically significant, it suggests a violation of RD assumptions.
• Covariate jumps imply that factors other than treatment may be driving the observed outcome change.

27.11.1.3 Manipulation Test (McCrary Density Test)

A critical assumption in RD is that units cannot precisely manipulate their values of $X_i$. If individuals can selectively sort around $c$ (e.g., students altering test scores to qualify for a scholarship), RD estimates become invalid.

To test for manipulation, we estimate the density function of $X$ and test for a discontinuity at $c$:

$$\hat{f}(X) = \lim_{x \uparrow c} f(X) - \lim_{x \downarrow c} f(X).$$

A significant difference suggests sorting behavior, which violates RD assumptions.

27.11.2 Functional Form of the Running Variable

27.11.2.1 General RD Model

The most flexible RD specification includes:

• A functional form $f(X_i)$ to account for trends.

• An indicator for treatment $D_i$. - An interaction term $D_i f(X_i)$ allowing for different slopes on each side.

$$Y_i = \alpha_0 + f(X_i) \alpha_1 + I(X_i \geq c) \alpha_2 + f(X_i) I(X_i \geq c) \alpha_3 + u_i.$$

where:

• $\alpha_2$ captures the treatment effect at $X = c$.

• $\alpha_3$ tests for differences in slopes across the threshold.

27.11.2.2 Simple Case: Linear RD

If $f(X_i)$ is a linear function, we estimate:

$$Y_i = \beta_0 + \beta_1 X_i + I(X_i \geq c) \beta_2 + \epsilon_i.$$

RD gives you $\beta_2$ (causal effect) of $X$ on $Y$ at the cutoff point

In practice, everyone does

$$Y_i = \alpha_0 + f(x) \alpha _1 + [I(x_i \ge c)]\alpha_2 + [f(x_i)\times [I(x_i \ge c)]\alpha_3 + u_i$$

where we estimate different slope on different sides of the line. And if you estimate $\alpha_3$ to be no different from 0 then we return to the simple case.

27.11.2.3 Higher-Order Polynomials

A more flexible specification allows for nonlinear relationships:

$$Y_i = \beta_0 + \beta_1 X_i + \beta_2 X_i^2 + \beta_3 X_i^3 + \tau D_i + \epsilon_i.$$

• Higher-order polynomials reduce bias but increase variance.
• Overfitting is a risk, especially with limited data near $c$.

27.11.2.4 Nonparametric Estimation

When polynomial models are too restrictive, we use nonparametric local linear regression:

$$\hat{E}[Y | X] = \sum_{i=1}^{n} K_h(X_i - c) Y_i.$$

where:

• $K_h(X)$ is a kernel function (e.g., Epanechnikov).

• $h$ is the bandwidth, chosen to optimize bias-variance tradeoff.

27.11.3 Bandwidth Selection

27.11.3.1 Tradeoff Between Bias and Efficiency

Choosing an appropriate bandwidth $h$ is crucial:

• Narrow $h$ (close to $c$): Lower bias, but high variance.

• Wider $h$: More efficient estimates but potential bias.

27.11.3.2 Optimal Bandwidth Selection

Several methods exist for selecting $h$:

1. Cross-validation: Minimizing mean squared error (MSE).
2. G. Imbens and Kalyanaraman (2012) bandwidth selection:
   • Balances bias-variance tradeoff.
   • Often reported in RD studies.
3. Cattaneo, Idrobo, and Titiunik (2019) robust bandwidth selection:
   • Focuses on valid inference, not just point estimation.

27.11.3.3 Bandwidth Sensitivity Analysis

A standard robustness check is estimating $\tau$ for different $h$ values:

• If results change drastically, estimates may be sensitive to bandwidth choice.
• If estimates remain stable, findings are more credible.

27.11.4 Addressing Potential Confounders

27.11.4.1 Multiple Running Variables

If multiple forcing variables influence treatment (e.g., both math and English scores for honors eligibility), failing to account for them may introduce confounding.

A solution is to extend RD to a multi-score framework:

$$Y_i = \alpha_0 + f(X_{1i}, X_{2i}) \alpha_1 + I(X_{1i} \geq c_1, X_{2i} \geq c_2) \alpha_2 + \epsilon_i.$$

• Controls for both scores simultaneously.
• Allows for interactions between assignment variables.

27.11.4.2 Bundling of Institutions

In cases where policies change at institutional levels (e.g., different states implementing varying minimum wages), treatment effects may reflect institutional bundling rather than individual cutoff effects.

One solution is to include institution fixed effects:

$$Y_i = \alpha + f(X_i) \beta + \tau D_i + \gamma C_j + \epsilon_i.$$

where $C_j$ is a categorical variable for institution $j$.

27.11.5 External Validity in RD

While RD provides strong internal validity, its generalizability is often limited:

1. Local Nature of RD Estimates
   • RD estimates apply only at the cutoff.
   • Effects may not extrapolate to other values of $X$.
2. Heterogeneous Treatment Effects
   • If $\tau$ varies across subgroups, RD estimates may not generalize.
3. Spillover Effects
   • If treatment effects extend beyond the threshold (e.g., peer effects), RD assumptions may be violated.

References

Cattaneo, Matias D, Nicolás Idrobo, and Rocı́o Titiunik. 2019. A Practical Introduction to Regression Discontinuity Designs: Foundations. Cambridge University Press.

Imbens, Guido, and Karthik Kalyanaraman. 2012. “Optimal Bandwidth Choice for the Regression Discontinuity Estimator.” The Review of Economic Studies 79 (3): 933–59.