title: Quasi-experimental study designs series-paper 6: risk of bias assessment.
abstract: OBJECTIVE
  Rigorous and transparent bias assessment is a core component of high-quality systematic reviews. We assess modifications to existing risk of bias approaches to incorporate rigorous quasi-experimental approaches with selection on unobservables. These are nonrandomized studies using design-based approaches to control for unobservable sources of confounding such as difference studies, instrumental variables, interrupted time series, natural experiments, and regression-discontinuity designs.
  
  
  METHODS
  We review existing risk of bias tools. Drawing on these tools, we present domains of bias and suggest directions for evaluation questions.
  
  
  RESULTS
  The review suggests that existing risk of bias tools provide, to different degrees, incomplete transparent criteria to assess the validity of these designs. The paper then presents an approach to evaluating the internal validity of quasi-experiments with selection on unobservables.
  
  
  CONCLUSIONS
  We conclude that tools for nonrandomized studies of interventions need to be further developed to incorporate evaluation questions for quasi-experiments with selection on unobservables.
authors.affiliation: International Initiative for Impact Evaluation, New Delhi, India. Electronic address: hwaddington@3ieimpact.org.
authors.forename: Hugh
authors.initials: H
authors.lastname: Waddington
authors.affiliation: University of Iowa, Iowa City, IA, USA.
authors.forename: Ariel M
authors.initials: AM
authors.lastname: Aloe
authors.affiliation: Florida State University, Tallahassee, FL, USA.
authors.forename: Betsy Jane
authors.initials: BJ
authors.lastname: Becker
authors.affiliation: International Initiative for Impact Evaluation, New Delhi, India.
authors.forename: Eric W
authors.initials: EW
authors.lastname: Djimeu
authors.affiliation: University of Sussex, Brighton, UK.
authors.forename: Jorge Garcia
authors.initials: JG
authors.lastname: Hombrados
authors.affiliation: Department of Medicine, University of Ottawa, Ottawa, Canada.
authors.forename: Peter
authors.initials: P
authors.lastname: Tugwell
authors.affiliation: Department of Medicine, University of Ottawa, Ottawa, Canada.
authors.forename: George
authors.initials: G
authors.lastname: Wells
authors.affiliation: University of Bristoln, Bristol, UK.
authors.forename: Barney
authors.initials: B
authors.lastname: Reeves
mesh_terms: D015982:Bias; D006801:Humans; D065228:Non-Randomized Controlled Trials as Topic / Q000706:statistics & numerical data*; D012107:Research Design; D018570:Risk Assessment
keywords: Difference in differences; Instrumental variables; Interrupted time series; Meta-Analysis; Natural experiment; Quasi-experiment; Regression discontinuity; Risk of bias; Systematic review