title: Estimating causal effects: considering three alternatives to difference-in-differences estimation.
abstract: Difference-in-differences (DiD) estimators provide unbiased treatment effect estimates when, in the absence of treatment, the average outcomes for the treated and control groups would have followed parallel trends over time. This assumption is implausible in many settings. An alternative assumption is that the potential outcomes are independent of treatment status, conditional on past outcomes. This paper considers three methods that share this assumption: the synthetic control method, a lagged dependent variable (LDV) regression approach, and matching on past outcomes. Our motivating empirical study is an evaluation of a hospital pay-for-performance scheme in England, the best practice tariffs programme. The conclusions of the original DiD analysis are sensitive to the choice of approach. We conduct a Monte Carlo simulation study that investigates these methods' performance. While DiD produces unbiased estimates when the parallel trends assumption holds, the alternative approaches provide less biased estimates of treatment effects when it is violated. In these cases, the LDV approach produces the most efficient and least biased estimates.
authors.affiliation: Department of Health Services Research and Policy, London School of Hygiene and Tropical Medicine, London, UK.
authors.forename: Stephen
authors.initials: S
authors.lastname: O'Neill
authors.affiliation: Department of Health Services Research and Policy, London School of Hygiene and Tropical Medicine, London, UK.
authors.forename: Noémi
authors.initials: N
authors.lastname: Kreif
authors.affiliation: Department of Health Services Research and Policy, London School of Hygiene and Tropical Medicine, London, UK.
authors.forename: Richard
authors.initials: R
authors.lastname: Grieve
authors.affiliation: Manchester Centre for Health Economics, Institute of Population Health, The University of Manchester, Manchester, UK.
authors.forename: Matthew
authors.initials: M
authors.lastname: Sutton
authors.affiliation: Department of Political Science and Department of Statistics, University of California at Berkeley, Berkeley, CA USA.
authors.forename: Jasjeet S
authors.initials: JS
authors.lastname: Sekhon
keywords: Difference-in-differences; Matching; Pay-for-performance; Policy evaluation; Synthetic control method