title: A reanalysis of cluster randomized trials showed interrupted time-series studies were valuable in health system evaluation.
abstract: OBJECTIVE
  There is often substantial uncertainty about the impacts of health system and policy interventions. Despite that, randomized controlled trials (RCTs) are uncommon in this field, partly because experiments can be difficult to carry out. An alternative method for impact evaluation is the interrupted time-series (ITS) design. Little is known, however, about how results from the two methods compare. Our aim was to explore whether ITS studies yield results that differ from those of randomized trials.
  
  
  METHODS
  We conducted single-arm ITS analyses (segmented regression) based on data from the intervention arm of cluster randomized trials (C-RCTs), that is, discarding control arm data. Secondarily, we included the control group data in the analyses, by subtracting control group data points from intervention group data points, thereby constructing a time series representing the difference between the intervention and control groups. We compared the results from the single-arm and controlled ITS analyses with results based on conventional aggregated analyses of trial data.
  
  
  RESULTS
  The findings were largely concordant, yielding effect estimates with overlapping 95% confidence intervals (CI) across different analytical methods. However, our analyses revealed the importance of a concurrent control group and of taking baseline and follow-up trends into account in the analysis of C-RCTs.
  
  
  CONCLUSIONS
  The ITS design is valuable for evaluation of health systems interventions, both when RCTs are not feasible and in the analysis and interpretation of data from C-RCTs.
authors.affiliation: Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, 133 Brookline Avenue, Boston, MA 02215, USA; Global Health Unit, Norwegian Knowledge Centre for the Health Services, PO Box 7004, St. Olavs pl, 0130 Oslo, Norway; Department of Community Medicine, Institute of Health and Society, University of Oslo, PO Box 1130 Blindern, 0318 Oslo, Norway. Electronic address: atle.fretheim@nokc.no.
authors.forename: Atle
authors.initials: A
authors.lastname: Fretheim
authors.affiliation: Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, 133 Brookline Avenue, Boston, MA 02215, USA.
authors.forename: Fang
authors.initials: F
authors.lastname: Zhang
authors.affiliation: Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, 133 Brookline Avenue, Boston, MA 02215, USA.
authors.forename: Dennis
authors.initials: D
authors.lastname: Ross-Degnan
authors.affiliation: Global Health Unit, Norwegian Knowledge Centre for the Health Services, PO Box 7004, St. Olavs pl, 0130 Oslo, Norway.
authors.forename: Andrew D
authors.initials: AD
authors.lastname: Oxman
authors.affiliation: Nursing Midwifery and Allied Health Professions Research Unit, University of Stirling, Stirling FK9 4LA, UK.
authors.forename: Helen
authors.initials: H
authors.lastname: Cheyne
authors.affiliation: Academic Unit of Primary Care, Leeds Institute of Health Sciences, University of Leeds, Charles Thackrah Building, 101 Clarendon Road, Leeds LS2 9LJ, UK.
authors.forename: Robbie
authors.initials: R
authors.lastname: Foy
authors.affiliation: School of Health and Related Research (ScHARR), University of Sheffield, Regent Court, Regent Street, Sheffield, S1 4DA, UK.
authors.forename: Steve
authors.initials: S
authors.lastname: Goodacre
authors.affiliation: Division of Cardiology, Yale University School of Medicine, Yale University, 333 Cedar St, New Haven, CT 06510, USA; Health Research & Educational Trust, 155 N Wacker, Suite 400, Chicago 60606, IL, USA.
authors.forename: Jeph
authors.initials: J
authors.lastname: Herrin
authors.affiliation: School of Population Health, University of Auckland, Private Bag 9201, Auckland, New Zealand.
authors.forename: Ngaire
authors.initials: N
authors.lastname: Kerse
authors.affiliation: Health Information Research Unit, Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada.
authors.forename: R James
authors.initials: RJ
authors.lastname: McKinlay
authors.affiliation: Department of Medicine, Brigham and Women's Hospital, 75 Francis Street, Boston, MA 02115, USA.
authors.forename: Adam
authors.initials: A
authors.lastname: Wright
authors.affiliation: Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, 133 Brookline Avenue, Boston, MA 02215, USA.
authors.forename: Stephen B
authors.initials: SB
authors.lastname: Soumerai
mesh_terms: D016000:Cluster Analysis; D003627:Data Interpretation, Statistical; D006302:Health Services Research / Q000592:standards / Q000706:statistics & numerical data*; D006801:Humans; D065186:Interrupted Time Series Analysis; D058996:Quality Improvement; D016032:Randomized Controlled Trials as Topic; D012107:Research Design
keywords: Evaluation methods; Health services research; Impact evaluations; Interrupted time-series; Quasi-experimental design; Randomized trials