title: Generalizability of findings from randomized controlled trials: application to the National Institute of Drug Abuse Clinical Trials Network.
abstract: OBJECTIVE
  To compare randomized controlled trial (RCT) sample treatment effects with the population effects of substance use disorder (SUD) treatment.
  
  
  METHODS
  Statistical weighting was used to re-compute the effects from 10 RCTs such that the participants in the trials had characteristics that resembled those of patients in the target populations.
  
  
  METHODS
  Multi-site RCTs and usual SUD treatment settings in the United States.
  
  
  METHODS
  A total of 3592 patients in 10 RCTs and 1 602 226 patients from usual SUD treatment settings between 2001 and 2009.
  
  
  METHODS
  Three outcomes of SUD treatment were examined: retention, urine toxicology and abstinence. We weighted the RCT sample treatment effects using propensity scores representing the conditional probability of participating in RCTs.
  
  
  RESULTS
  Weighting the samples changed the significance of estimated sample treatment effects. Most commonly, positive effects of trials became statistically non-significant after weighting (three trials for retention and urine toxicology and one trial for abstinence); also, non-significant effects became significantly positive (one trial for abstinence) and significantly negative effects became non-significant (two trials for abstinence). There was suggestive evidence of treatment effect heterogeneity in subgroups that are under- or over-represented in the trials, some of which were consistent with the differences in average treatment effects between weighted and unweighted results.
  
  
  CONCLUSIONS
  The findings of randomized controlled trials (RCTs) for substance use disorder treatment do not appear to be directly generalizable to target populations when the RCT samples do not reflect adequately the target populations and there is treatment effect heterogeneity across patient subgroups.
authors.affiliation: Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.
authors.forename: Ryoko
authors.initials: R
authors.lastname: Susukida
authors.affiliation: Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.|Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.|Department of Psychiatry and Behavioral Sciences, Johns Hopkins University School of Medicine, Baltimore, MD, USA.
authors.forename: Rosa M
authors.initials: RM
authors.lastname: Crum
authors.affiliation: Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.|The Fred Hutchinson Cancer Research Center, Seattle, WA, USA.
authors.forename: Cyrus
authors.initials: C
authors.lastname: Ebnesajjad
authors.affiliation: Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.|Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.|Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.
authors.forename: Elizabeth A
authors.initials: EA
authors.lastname: Stuart
authors.affiliation: Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.|Department of Psychiatry and Behavioral Sciences, Johns Hopkins University School of Medicine, Baltimore, MD, USA.
authors.forename: Ramin
authors.initials: R
authors.lastname: Mojtabai
mesh_terms: D000293:Adolescent; D000328:Adult; D005260:Female; D006801:Humans; D008297:Male; D016032:Randomized Controlled Trials as Topic / Q000379:methods* / Q000706:statistics & numerical data*; D019966:Substance-Related Disorders / Q000628:therapy*; D016896:Treatment Outcome; D014481:United States; D055815:Young Adult
keywords: Generalizability; National Institute of Drug Abuse Clinical Trials Network; propensity score weighting; randomized controlled trials; substance use disorder treatment; treatment effect heterogeneity