26.2 Robustness Checks

Robustness checks are essential to demonstrate that findings are not driven by model specification choices.

Recommended robustness checks (Goldfarb, Tucker, and Wang 2022):

1. Alternative Control Sets
   • Show results with and without controls.
   • Examine how the estimate of interest changes.
   • Use Rosenbaum bounds for formal sensitivity analysis (Altonji, Elder, and Taber 2005).
   • In marketing applications, see (Manchanda, Packard, and Pattabhiramaiah 2015) and (Shin, Sudhir, and Yoon 2012).
2. Different Functional Forms
   • Check whether the results hold under different model specifications (e.g., linear vs. non-linear models).
3. Varying Time Windows
   • In longitudinal settings, test different time frames to ensure robustness.
4. Alternative Dependent Variables
   • Use related outcomes or different measures of the dependent variable.
5. Varying Control Group Size
   • Compare results using matched vs. unmatched samples to assess sensitivity to sample selection.
6. Placebo Tests
   • Conduct placebo tests to ensure the effect is not spurious.
   • The appropriate placebo test depends on the specific quasi-experimental method used (examples provided in later sections).

---

References

Altonji, Joseph G, Todd E Elder, and Christopher R Taber. 2005. “Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools.” Journal of Political Economy 113 (1): 151–84.

Goldfarb, Avi, Catherine Tucker, and Yanwen Wang. 2022. “Conducting Research in Marketing with Quasi-Experiments.” Journal of Marketing 86 (3): 1–20.

Manchanda, Puneet, Grant Packard, and Adithya Pattabhiramaiah. 2015. “Social Dollars: The Economic Impact of Customer Participation in a Firm-Sponsored Online Customer Community.” Marketing Science 34 (3): 367–87.

Shin, Jiwoong, K Sudhir, and Dae-Hee Yoon. 2012. “When to ‘Fire’ Customers: Customer Cost-Based Pricing.” Management Science 58 (5): 932–47.