title: Problems caused by heterogeneity in meta-analysis: a case study of acupuncture trials.
abstract: OBJECTIVE
  To illustrate the pitfalls of using meta-analysis to combine estimates of effect in trials that are highly varied and have a high potential for bias.
  
  
  METHODS
  We used a random-effects meta-analysis to pool the results of 51 sham-controlled acupuncture trials of chronic pain published in English before 2008 and explored the heterogeneity using meta-regression. We repeated the process on a subset of these trials that used a visually credible non-penetrating sham device as control (N = 12).
  
  
  RESULTS
  In both analyses there were high levels of heterogeneity and many studies were at risk from potential bias. The heterogeneity was not explained by meta-regression.
  
  
  CONCLUSIONS
  Trials of interventions that have high potential for bias, such as many in the acupuncture literature, do not meet the assumptions of the statistical procedure that underlie random-effects meta-analysis. Even in the absence of bias, heterogeneity in meta-analyses is not accounted for by the CIs around the pooled estimate.
authors.affiliation: Department of Health Sciences, The University of York, , York, UK.
authors.forename: Stephanie L
authors.initials: SL
authors.lastname: Prady
authors.forename: Jane
authors.initials: J
authors.lastname: Burch
authors.forename: Simon
authors.initials: S
authors.lastname: Crouch
authors.forename: Hugh
authors.initials: H
authors.lastname: MacPherson
mesh_terms: D015670:Acupuncture Therapy; D006801:Humans; D015201:Meta-Analysis as Topic; D016032:Randomized Controlled Trials as Topic; D012044:Regression Analysis; D012107:Research Design / Q000592:standards*
keywords: Acupuncture; Statistics & Research Methods; Systematic Reviews