title: Inconsistency between direct and indirect comparisons of competing interventions: meta-epidemiological study.
abstract: OBJECTIVE
  To investigate the agreement between direct and indirect comparisons of competing healthcare interventions.
  
  
  METHODS
  Meta-epidemiological study based on sample of meta-analyses of randomised controlled trials. Data sources Cochrane Database of Systematic Reviews and PubMed. Inclusion criteria Systematic reviews that provided sufficient data for both direct comparison and independent indirect comparisons of two interventions on the basis of a common comparator and in which the odds ratio could be used as the outcome statistic.
  
  
  METHODS
  Inconsistency measured by the difference in the log odds ratio between the direct and indirect methods.
  
  
  RESULTS
  The study included 112 independent trial networks (including 1552 trials with 478,775 patients in total) that allowed both direct and indirect comparison of two interventions. Indirect comparison had already been explicitly done in only 13 of the 85 Cochrane reviews included. The inconsistency between the direct and indirect comparison was statistically significant in 16 cases (14%, 95% confidence interval 9% to 22%). The statistically significant inconsistency was associated with fewer trials, subjectively assessed outcomes, and statistically significant effects of treatment in either direct or indirect comparisons. Owing to considerable inconsistency, many (14/39) of the statistically significant effects by direct comparison became non-significant when the direct and indirect estimates were combined.
  
  
  CONCLUSIONS
  Significant inconsistency between direct and indirect comparisons may be more prevalent than previously observed. Direct and indirect estimates should be combined in mixed treatment comparisons only after adequate assessment of the consistency of the evidence.
authors.affiliation: Faculty of Medicine and Health Science, Norwich Medical School, University of East Anglia, Norwich NR4 7TJ, UK. fujian.song@uea.ac.uk
authors.forename: Fujian
authors.initials: F
authors.lastname: Song
authors.forename: Tengbin
authors.initials: T
authors.lastname: Xiong
authors.forename: Sheetal
authors.initials: S
authors.lastname: Parekh-Bhurke
authors.forename: Yoon K
authors.initials: YK
authors.lastname: Loke
authors.forename: Alex J
authors.initials: AJ
authors.lastname: Sutton
authors.forename: Alison J
authors.initials: AJ
authors.lastname: Eastwood
authors.forename: Richard
authors.initials: R
authors.lastname: Holland
authors.forename: Yen-Fu
authors.initials: YF
authors.lastname: Chen
authors.forename: Anne-Marie
authors.initials: AM
authors.lastname: Glenny
authors.forename: Jonathan J
authors.initials: JJ
authors.lastname: Deeks
authors.forename: Doug G
authors.initials: DG
authors.lastname: Altman
mesh_terms: D015982:Bias; D057186:Comparative Effectiveness Research; D016021:Epidemiologic Studies; D006801:Humans; D015201:Meta-Analysis as Topic; D016032:Randomized Controlled Trials as Topic; D015203:Reproducibility of Results; D012107:Research Design; D012196:Review Literature as Topic; D016896:Treatment Outcome