title: The impact of study size on meta-analyses: examination of underpowered studies in Cochrane reviews.
abstract: BACKGROUND
  Most meta-analyses include data from one or more small studies that, individually, do not have power to detect an intervention effect. The relative influence of adequately powered and underpowered studies in published meta-analyses has not previously been explored. We examine the distribution of power available in studies within meta-analyses published in Cochrane reviews, and investigate the impact of underpowered studies on meta-analysis results.
  
  
  RESULTS
  For 14,886 meta-analyses of binary outcomes from 1,991 Cochrane reviews, we calculated power per study within each meta-analysis. We defined adequate power as ≥50% power to detect a 30% relative risk reduction. In a subset of 1,107 meta-analyses including 5 or more studies with at least two adequately powered and at least one underpowered, results were compared with and without underpowered studies. In 10,492 (70%) of 14,886 meta-analyses, all included studies were underpowered; only 2,588 (17%) included at least two adequately powered studies. 34% of the meta-analyses themselves were adequately powered. The median of summary relative risks was 0.75 across all meta-analyses (inter-quartile range 0.55 to 0.89). In the subset examined, odds ratios in underpowered studies were 15% lower (95% CI 11% to 18%, P<0.0001) than in adequately powered studies, in meta-analyses of controlled pharmacological trials; and 12% lower (95% CI 7% to 17%, P<0.0001) in meta-analyses of controlled non-pharmacological trials. The standard error of the intervention effect increased by a median of 11% (inter-quartile range -1% to 35%) when underpowered studies were omitted; and between-study heterogeneity tended to decrease.
  
  
  CONCLUSIONS
  When at least two adequately powered studies are available in meta-analyses reported by Cochrane reviews, underpowered studies often contribute little information, and could be left out if a rapid review of the evidence is required. However, underpowered studies made up the entirety of the evidence in most Cochrane reviews.
authors.affiliation: MRC Biostatistics Unit, Institute of Public Health, Cambridge, United Kingdom. rebecca.turner@mrc-bsu.cam.ac.uk
authors.forename: Rebecca M
authors.initials: RM
authors.lastname: Turner
authors.forename: Sheila M
authors.initials: SM
authors.lastname: Bird
authors.forename: Julian P T
authors.initials: JP
authors.lastname: Higgins
mesh_terms: D006801:Humans; D015201:Meta-Analysis as Topic; D016017:Odds Ratio; D012196:Review Literature as Topic; D040242:Risk Reduction Behavior; D018401:Sample Size
references.citation: Control Clin Trials. 1986 Sep;7(3):177-88
references.pmid: 3802833
references.citation: BMC Med Res Methodol. 2011 Nov 24;11:160
references.pmid: 22114982
references.citation: J Clin Epidemiol. 2008 Jan;61(1):64-75
references.pmid: 18083463
references.citation: Lancet. 1997 Sep 13;350(9080):804-7
references.pmid: 9298015
references.citation: PLoS One. 2011;6(10):e25491
references.pmid: 22028777
references.citation: BMJ. 2010 Jul 16;341:c3515
references.pmid: 20639294
references.citation: BMJ. 1994 Jan 29;308(6924):283-4
references.pmid: 8124111
references.citation: BMJ. 2009 May 12;338:b1732
references.pmid: 19435763
references.citation: Int J Technol Assess Health Care. 2008 Spring;24(2):133-9
references.pmid: 18400114
references.citation: JAMA. 2002 Jul 17;288(3):358-62
references.pmid: 12117401
references.citation: Contemp Clin Trials. 2007 May;28(3):324-8
references.pmid: 17188025
references.citation: J Clin Epidemiol. 2008 Aug;61(8):763-9
references.pmid: 18411040
references.citation: Ann Intern Med. 2001 Dec 4;135(11):982-9
references.pmid: 11730399
references.citation: Psychol Methods. 2001 Sep;6(3):203-17
references.pmid: 11570228
references.citation: J Clin Epidemiol. 2000 Nov;53(11):1119-29
references.pmid: 11106885
references.citation: J Clin Epidemiol. 2009 Jul;62(7):711-715.e1
references.pmid: 19171462
references.citation: BMJ. 2011 Jul 22;343:d4002
references.pmid: 21784880
references.citation: PLoS Med. 2008 Jan 3;5(1):e4
references.pmid: 18177203
references.citation: J R Stat Soc Ser A Stat Soc. 2009 Jan;172(1):21-47
references.pmid: 19381328
references.citation: JAMA. 1995 Nov 15;274(19):1526-33
references.pmid: 7474221
references.citation: Biom J. 2011 Mar;53(2):351-68
references.pmid: 21374698
references.citation: Addiction. 2010 Sep;105(9):1545-54
references.pmid: 20579009
references.citation: BMC Med Res Methodol. 2009 Jan 12;9:2
references.pmid: 19138428