title: Random error in cardiovascular meta-analyses: how common are false positive and false negative results?
abstract: BACKGROUND
  Cochrane reviews are viewed as the gold standard in meta-analyses given their efforts to identify and limit systematic error which could cause spurious conclusions. The potential for random error to cause spurious conclusions in meta-analyses is less well appreciated.
  
  
  METHODS
  We examined all reviews approved and published by the Cochrane Heart Group in the 2012 Cochrane Library that included at least one meta-analysis with 5 or more randomized trials. We used trial sequential analysis to classify statistically significant meta-analyses as true positives if their pooled sample size and/or their cumulative Z-curve crossed the O'Brien-Fleming monitoring boundaries for detecting a RRR of at least 25%. We classified meta-analyses that did not achieve statistical significance as true negatives if their pooled sample size was sufficient to reject a RRR of 25%.
  
  
  RESULTS
  Twenty three (41%) of the 56 meta-analyses reported statistically significant results, and 19 (83%) were true positives. Of the 33 non-statistically significant meta-analyses, 12 (36%) were true negatives. Overall, 25 (45%) of the 56 published Cochrane reviews were too small to detect/rule out an effect size of at least 25% - 12 were acknowledged as such by their authors. Of the 22 meta-analyses which were reported to be conclusive by their authors, 12 (55%) contained insufficient data to detect/rule out a 25% relative treatment effect.
  
  
  CONCLUSIONS
  False positive and false negative meta-analyses are common but infrequently recognized, even among methodologically robust reviews published by the Cochrane Heart Group. Meta-analysts and readers should incorporate trial sequential analysis when interpreting results.
authors.affiliation: The Division of General Internal Medicine, University of Alberta, Edmonton, Canada.
authors.forename: Zaina
authors.initials: Z
authors.lastname: AlBalawi
authors.forename: Finlay A
authors.initials: FA
authors.lastname: McAlister
authors.forename: Kristian
authors.initials: K
authors.lastname: Thorlund
authors.forename: Michelle
authors.initials: M
authors.lastname: Wong
authors.forename: Jørn
authors.initials: J
authors.lastname: Wetterslev
mesh_terms: D002318:Cardiovascular Diseases / Q000175:diagnosis* / Q000453:epidemiology*; D005188:False Negative Reactions; D005189:False Positive Reactions; D016032:Randomized Controlled Trials as Topic / Q000706:statistics & numerical data; D012107:Research Design / Q000592:standards* / Q000706:statistics & numerical data*; D018401:Sample Size