title: Inclusion of quasi-experimental studies in systematic reviews of health systems research.
abstract: Systematic reviews of health systems research commonly limit studies for evidence synthesis to randomized controlled trials. However, well-conducted quasi-experimental studies can provide strong evidence for causal inference. With this article, we aim to stimulate and inform discussions on including quasi-experiments in systematic reviews of health systems research. We define quasi-experimental studies as those that estimate causal effect sizes using exogenous variation in the exposure of interest that is not directly controlled by the researcher. We incorporate this definition into a non-hierarchical three-class taxonomy of study designs - experiments, quasi-experiments, and non-experiments. Based on a review of practice in three disciplines related to health systems research (epidemiology, economics, and political science), we discuss five commonly used study designs that fit our definition of quasi-experiments: natural experiments, instrumental variable analyses, regression discontinuity analyses, interrupted times series studies, and difference studies including controlled before-and-after designs, difference-in-difference designs and fixed effects analyses of panel data. We further review current practices regarding quasi-experimental studies in three non-health fields that utilize systematic reviews (education, development, and environment studies) to inform the design of approaches for synthesizing quasi-experimental evidence in health systems research. Ultimately, the aim of any review is practical: to provide useful information for policymakers, practitioners, and researchers. Future work should focus on building a consensus among users and producers of systematic reviews regarding the inclusion of quasi-experiments.
authors.affiliation: Department of Global Health, Boston University School of Public Health, Boston, USA; Center for Global Health and Development, Boston University, Boston, USA.
authors.forename: Peter C
authors.initials: PC
authors.lastname: Rockers
authors.affiliation: Norwegian Institute of Public Health, Oslo, Norway; Department of Global Health and Population, Harvard T.H. Chan School of Public Health, Boston, USA; Department of Health Management and Health Economics, Institute for Health and Society, University of Oslo, Norway; Harvard Global Health Institute, Harvard University, Cambridge, USA.
authors.forename: John-Arne
authors.initials: JA
authors.lastname: Røttingen
authors.affiliation: Behavior and Health Research Unit, University of Cambridge, Cambridge, UK.
authors.forename: Ian
authors.initials: I
authors.lastname: Shemilt
authors.affiliation: Department of Medicine, University of Ottawa, Canada; Department and Epidemiology and Community Health, University of Ottawa, Canada.
authors.forename: Peter
authors.initials: P
authors.lastname: Tugwell
authors.affiliation: Department of Global Health and Population, Harvard T.H. Chan School of Public Health, Boston, USA; Wellcome Trust Africa Centre for Health and Population Studies, University of KwaZulu-Natal, Mtubatuba, South Africa. Electronic address: tbaernig@hsph.harvard.edu.
authors.forename: Till
authors.initials: T
authors.lastname: Bärnighausen
mesh_terms: D055317:Evidence-Based Practice; D006302:Health Services Research; D012044:Regression Analysis; D012107:Research Design; D012196:Review Literature as Topic
keywords: Difference-in-difference studies; Instrumental variable analysis; Interrupted time series; Quasi-experimental studies; Regression discontinuity designs; Systematic reviews