title: The Validity of the Comparative Interrupted Time Series Design for Evaluating the Effect of School-Level Interventions.
abstract: In this article, we examine whether a well-executed comparative interrupted time series (CITS) design can produce valid inferences about the effectiveness of a school-level intervention. This article also explores the trade-off between bias reduction and precision loss across different methods of selecting comparison groups for the CITS design and assesses whether choosing matched comparison schools based only on preintervention test scores is sufficient to produce internally valid impact estimates.
  
  
  
  We conduct a validation study of the CITS design based on the federal Reading First program as implemented in one state using results from a regression discontinuity design as a causal benchmark.
  
  
  
  Our results contribute to the growing base of evidence regarding the validity of nonexperimental designs. We demonstrate that the CITS design can, in our example, produce internally valid estimates of program impacts when multiple years of preintervention outcome data (test scores in the present case) are available and when a set of reasonable criteria are used to select comparison organizations (schools in the present case).
authors.affiliation: Institute for Social Research, University of Michigan, Ann Arbor, MI, USA rjacob@umich.edu.
authors.forename: Robin
authors.initials: R
authors.lastname: Jacob
authors.affiliation: MDRC, New York, NY, USA.
authors.forename: Marie-Andree
authors.initials: MA
authors.lastname: Somers
authors.affiliation: MDRC, New York, NY, USA.
authors.forename: Pei
authors.initials: P
authors.lastname: Zhu
authors.affiliation: MDRC, New York, NY, USA.
authors.forename: Howard
authors.initials: H
authors.lastname: Bloom
mesh_terms: D005260:Female; D006801:Humans; D065186:Interrupted Time Series Analysis; D008297:Male; D012107:Research Design; D012574:Schools / Q000458:organization & administration*; D014481:United States
keywords: comparative interrupted time series analysis; design replication; quasi-experimental methods; validation study; within-study comparison