title: How does under-reporting of negative and inconclusive results affect the false-positive rate in meta-analysis? A simulation study.
abstract: OBJECTIVE
  To investigate the impact of a higher publishing probability for statistically significant positive outcomes on the false-positive rate in meta-analysis.
  
  
  METHODS
  Meta-analyses of different sizes (N=10, N=20, N=50 and N=100), levels of heterogeneity and levels of publication bias were simulated.
  
  
  METHODS
  The type I error rate for the test of the mean effect size (ie, the rate at which the meta-analyses showed that the mean effect differed from 0 when it in fact equalled 0) was estimated. Additionally, the power and type I error rate of publication bias detection methods based on the funnel plot were estimated.
  
  
  RESULTS
  In the presence of a publication bias characterised by a higher probability of including statistically significant positive results, the meta-analyses frequently concluded that the mean effect size differed from zero when it actually equalled zero. The magnitude of the effect of publication bias increased with an increasing number of studies and between-study variability. A higher probability of including statistically significant positive outcomes introduced little asymmetry to the funnel plot. A publication bias of a sufficient magnitude to frequently overturn the meta-analytic conclusions was difficult to detect by publication bias tests based on the funnel plot. When statistically significant positive results were four times more likely to be included than other outcomes and a large between-study variability was present, more than 90% of the meta-analyses of 50 and 100 studies wrongly showed that the mean effect size differed from zero. In the same scenario, publication bias tests based on the funnel plot detected the bias at rates not exceeding 15%.
  
  
  CONCLUSIONS
  This study adds to the evidence that publication bias is a major threat to the validity of medical research and supports the usefulness of efforts to limit publication bias.
authors.affiliation: Department of Science, Hasselt University, Diepenbeek, Belgium.
authors.forename: Michal
authors.initials: M
authors.lastname: Kicinski
mesh_terms: D035843:Biomedical Research / Q000706:statistics & numerical data*; D005189:False Positive Reactions; D006801:Humans; D015201:Meta-Analysis as Topic; D017594:Publication Bias
keywords: EGGER'S TEST; FUNNEL PLOT; META-ANALYSIS; PUBLICATION BIAS; TYPE I ERROR