26.1 Identification Strategy in Quasi-Experiments

Unlike randomized experiments, quasi-experiments lack formal statistical proof of causality. Instead, researchers must build a plausible argument supported by empirical evidence.

Key components of an identification strategy:

Source of Exogenous Variation
- Justify where the exogenous variation originates.
- Use institutional knowledge and theoretical arguments to support this claim.
Exclusion Restriction
- Provide evidence that variation in the exogenous shock affects the outcome only through the proposed mechanism.
- This requires ruling out confounding factors.
Stable Unit Treatment Value Assumption
- The treatment of unit $i$ should only affect the outcome of unit $i$ .
- No spillovers or interference between treatment and control groups.

Every quasi-experimental method involves a tradeoff between statistical power and support for the exogeneity assumption. This means that researchers often discard variation in the data that does not meet the exogeneity assumption.

Important Notes:

$R^2$ is not a reliable metric in causal inference and can be misleading for model comparison (Ebbes, Papies, and Van Heerde 2011).
Clustering should be determined based on the study design, not just expectations of correlation (Abadie et al. 2023).
For small samples, use the wild bootstrap procedure to correct for downward bias (Cameron, Gelbach, and Miller 2008). See also (Cai et al. 2022) for further assumptions.

References

Abadie, Alberto, Susan Athey, Guido W Imbens, and Jeffrey M Wooldridge. 2023. “When Should You Adjust Standard Errors for Clustering?” The Quarterly Journal of Economics 138 (1): 1–35.

Cai, Yong, Ivan A Canay, Deborah Kim, and Azeem M Shaikh. 2022. “On the Implementation of Approximate Randomization Tests in Linear Models with a Small Number of Clusters.” Journal of Econometric Methods 12 (1): 85–103.

Cameron, A Colin, Jonah B Gelbach, and Douglas L Miller. 2008. “Bootstrap-Based Improvements for Inference with Clustered Errors.” The Review of Economics and Statistics 90 (3): 414–27.

Ebbes, Peter, Dominik Papies, and Harald J Van Heerde. 2011. “The Sense and Non-Sense of Holdout Sample Validation in the Presence of Endogeneity.” Marketing Science 30 (6): 1115–22.