title: Evaluation of Evidence of Statistical Support and Corroboration of Subgroup Claims in Randomized Clinical Trials.
abstract: Many published randomized clinical trials (RCTs) make claims for subgroup differences.
  
  
  
  To evaluate how often subgroup claims reported in the abstracts of RCTs are actually supported by statistical evidence (P < .05 from an interaction test) and corroborated by subsequent RCTs and meta-analyses.
  
  
  
  This meta-epidemiological survey examines data sets of trials with at least 1 subgroup claim, including Subgroup Analysis of Trials Is Rarely Easy (SATIRE) articles and Discontinuation of Randomized Trials (DISCO) articles. We used Scopus (updated July 2016) to search for English-language articles citing each of the eligible index articles with at least 1 subgroup finding in the abstract.
  
  
  
  Articles with a subgroup claim in the abstract with or without evidence of statistical heterogeneity (P < .05 from an interaction test) in the text and articles attempting to corroborate the subgroup findings.
  
  
  
  Study characteristics of trials with at least 1 subgroup claim in the abstract were recorded. Two reviewers extracted the data necessary to calculate subgroup-level effect sizes, standard errors, and the P values for interaction. For individual RCTs and meta-analyses that attempted to corroborate the subgroup findings from the index articles, trial characteristics were extracted. Cochran Q test was used to reevaluate heterogeneity with the data from all available trials.
  
  
  
  The number of subgroup claims in the abstracts of RCTs, the number of subgroup claims in the abstracts of RCTs with statistical support (subgroup findings), and the number of subgroup findings corroborated by subsequent RCTs and meta-analyses.
  
  
  
  Sixty-four eligible RCTs made a total of 117 subgroup claims in their abstracts. Of these 117 claims, only 46 (39.3%) in 33 articles had evidence of statistically significant heterogeneity from a test for interaction. In addition, out of these 46 subgroup findings, only 16 (34.8%) ensured balance between randomization groups within the subgroups (eg, through stratified randomization), 13 (28.3%) entailed a prespecified subgroup analysis, and 1 (2.2%) was adjusted for multiple testing. Only 5 (10.9%) of the 46 subgroup findings had at least 1 subsequent pure corroboration attempt by a meta-analysis or an RCT. In all 5 cases, the corroboration attempts found no evidence of a statistically significant subgroup effect. In addition, all effect sizes from meta-analyses were attenuated toward the null.
  
  
  
  A minority of subgroup claims made in the abstracts of RCTs are supported by their own data (ie, a significant interaction effect). For those that have statistical support (P < .05 from an interaction test), most fail to meet other best practices for subgroup tests, including prespecification, stratified randomization, and adjustment for multiple testing. Attempts to corroborate statistically significant subgroup differences are rare; when done, the initially observed subgroup differences are not reproduced.
authors.affiliation: Department of Health Research and Policy, Stanford University School of Medicine, Stanford, California2Meta-Research Innovation Center at Stanford (METRICS), Stanford University School of Medicine, Stanford, California.
authors.forename: Joshua D
authors.initials: JD
authors.lastname: Wallach
authors.affiliation: Department of Health Research and Policy, Stanford University School of Medicine, Stanford, California2Meta-Research Innovation Center at Stanford (METRICS), Stanford University School of Medicine, Stanford, California3Department of Medicine, Stanford University School of Medicine, Stanford, California.
authors.forename: Patrick G
authors.initials: PG
authors.lastname: Sullivan
authors.affiliation: Stanford Prevention Research Center, Department of Medicine, Stanford University School of Medicine, Stanford, California.
authors.forename: John F
authors.initials: JF
authors.lastname: Trepanowski
authors.affiliation: Department of Health Research and Policy, Stanford University School of Medicine, Stanford, California.
authors.forename: Kristin L
authors.initials: KL
authors.lastname: Sainani
authors.affiliation: Department of Public Health, Erasmus MC, Rotterdam, the Netherlands.
authors.forename: Ewout W
authors.initials: EW
authors.lastname: Steyerberg
authors.affiliation: Department of Health Research and Policy, Stanford University School of Medicine, Stanford, California2Meta-Research Innovation Center at Stanford (METRICS), Stanford University School of Medicine, Stanford, California3Department of Medicine, Stanford University School of Medicine, Stanford, California4Stanford Prevention Research Center, Department of Medicine, Stanford University School of Medicine, Stanford, California6Department of Statistics, Stanford University School of Humanities and Sciences, Stanford, California.
authors.forename: John P A
authors.initials: JP
authors.lastname: Ioannidis
mesh_terms: D003627:Data Interpretation, Statistical; D003953:Diagnosis-Related Groups; D019317:Evidence-Based Medicine / Q000379:methods / Q000592:standards; D006801:Humans; D057194:Intention to Treat Analysis; D011643:Publishing / Q000592:standards*; D016032:Randomized Controlled Trials as Topic / Q000379:methods / Q000592:standards