title: Dicing with chance, life and death in systematic reviews and meta-analyses: D.I.C.E. 3, a simulation study.
abstract: OBJECTIVE
  To show the effects of chance on meta-analyses, and the potential dangers of being prompted to do a meta-analysis by one favourable trial.
  
  
  METHODS
  In total, 100,000 trials were simulated and combined into 10,000 meta-analyses, using data from the control group of a cancer trial. Each participant record was randomly coded to simulate allocation to 'treatment' or 'control'.
  
  
  METHODS
  Simulated study.
  
  
  METHODS
  De-identified records for 578 patients from the control group of a cancer trial, of whom 147 had died.
  
  
  METHODS
  Time to death from any cause.
  
  
  RESULTS
  Of the 100,000 trials, 4897 (4.9%) were statistically significant at 2p < 0.05 and 123 (1.2%) of the 10,000 meta-analyses were significant at 2p < 0.01. The most extreme result was a 20% reduction (99% CI: 0.70-0.91; 2p = 0.00002) in the annual odds of dying in the 'treatment' group. If a meta-analysis contained at least one trial with a statistically significant result (at 2p < 0.05), the likelihood of the meta-analysis being significant (at 2p < 0.01) increased strikingly. For example, among the 473 meta-analyses in which the first trial in a batch of 10 was statistically significant (at 2p < 0.05), 18 (3.8%) favoured treatment at 2p < 0.01.
  
  
  CONCLUSIONS
  Chance can influence the results of meta-analyses regardless of how well they are conducted. Researchers should not ignore this when they plan a meta-analysis and when they report their results. People reading their reports should also be wary. Caution is particularly important when the results of one or more included studies influenced the decision to do the meta-analysis.
authors: All-Ireland Hub for Trials Methodology Research, Centre for Public Health, Institute of Clinical Sciences, Queens University Belfast, Royal Hospitals, Belfast BT12 6BJ, UK.
authors: Mike
authors: M
authors: Clarke
authors: Jim
authors: J
authors: Halsey
mesh_terms: D015982:Bias; D001699:Biometry; D002986:Clinical Trials as Topic; D003198:Computer Simulation; D006801:Humans; D015201:Meta-Analysis as Topic; D017063:Outcome Assessment, Health Care; D012196:Review Literature as Topic
keywords: Systematic reviews; chance; meta-analysis; randomized trials; statistical analysis