title: Predicting the extent of heterogeneity in meta-analysis, using empirical data from the Cochrane Database of Systematic Reviews.
abstract: BACKGROUND
  Many meta-analyses contain only a small number of studies, which makes it difficult to estimate the extent of between-study heterogeneity. Bayesian meta-analysis allows incorporation of external evidence on heterogeneity, and offers advantages over conventional random-effects meta-analysis. To assist in this, we provide empirical evidence on the likely extent of heterogeneity in particular areas of health care.
  
  
  METHODS
  Our analyses included 14 886 meta-analyses from the Cochrane Database of Systematic Reviews. We classified each meta-analysis according to the type of outcome, type of intervention comparison and medical specialty. By modelling the study data from all meta-analyses simultaneously, using the log odds ratio scale, we investigated the impact of meta-analysis characteristics on the underlying between-study heterogeneity variance. Predictive distributions were obtained for the heterogeneity expected in future meta-analyses.
  
  
  RESULTS
  Between-study heterogeneity variances for meta-analyses in which the outcome was all-cause mortality were found to be on average 17% (95% CI 10-26) of variances for other outcomes. In meta-analyses comparing two active pharmacological interventions, heterogeneity was on average 75% (95% CI 58-95) of variances for non-pharmacological interventions. Meta-analysis size was found to have only a small effect on heterogeneity. Predictive distributions are presented for nine different settings, defined by type of outcome and type of intervention comparison. For example, for a planned meta-analysis comparing a pharmacological intervention against placebo or control with a subjectively measured outcome, the predictive distribution for heterogeneity is a log-normal (-2.13, 1.58(2)) distribution, which has a median value of 0.12. In an example of meta-analysis of six studies, incorporating external evidence led to a smaller heterogeneity estimate and a narrower confidence interval for the combined intervention effect.
  
  
  CONCLUSIONS
  Meta-analysis characteristics were strongly associated with the degree of between-study heterogeneity, and predictive distributions for heterogeneity differed substantially across settings. The informative priors provided will be very beneficial in future meta-analyses including few studies.
authors.affiliation: MRC Biostatistics Unit, Institute of Public Health, Cambridge, UK. rebecca.turner@mrc-bsu.cam.ac.uk
authors.forename: Rebecca M
authors.initials: RM
authors.lastname: Turner
authors.forename: Jonathan
authors.initials: J
authors.lastname: Davey
authors.forename: Mike J
authors.initials: MJ
authors.lastname: Clarke
authors.forename: Simon G
authors.initials: SG
authors.lastname: Thompson
authors.forename: Julian Pt
authors.initials: JP
authors.lastname: Higgins
mesh_terms: D001499:Bayes Theorem; D006801:Humans; D015201:Meta-Analysis as Topic; D017063:Outcome Assessment, Health Care / Q000379:methods; D012107:Research Design