title: Quantifying the risk of error when interpreting funnel plots.
abstract: BACKGROUND
  Funnel plots are widely used to investigate possible publication bias in meta-analyses. There has, however, been little formal assessment of whether a visual inspection of a funnel plot is sufficient to identify publication bias.
  
  
  METHODS
  Visual assessment of bias in a funnel plot is quantified using two new statistics: the Imbalance and the Asymmetry Distance, both intended to replicate how a funnel plot is typically assessed. A simulation study was performed to assess the performance of these two statistics for identifying publication bias.
  
  
  RESULTS
  The two statistics both have high type I error and low statistical power, unless the number of studies in the meta-analysis is very large. These results suggest that visual inspection of a funnel plot is unlikely to lead to a valid assessment of publication bias.
  
  
  CONCLUSIONS
  In most systematic reviews, visual inspection of a funnel plot may give a misleading impression of the presence or absence of publication bias. Formal statistical tests for bias should generally be preferred.
authors: Centre for Reviews and Dissemination, University of York, Heslington, York, YO10 5DD, UK. mark.simmonds@york.ac.uk.
authors: Mark
authors: M
authors: Simmonds
mesh_terms: D003198:Computer Simulation; D006801:Humans; D015201:Meta-Analysis as Topic; D017594:Publication Bias / Q000706:statistics & numerical data*; D012306:Risk