title: Evaluation of inconsistency in networks of interventions.
abstract: BACKGROUND
  The assumption of consistency, defined as agreement between direct and indirect sources of evidence, underlies the increasingly popular method of network meta-analysis. No evidence exists so far regarding the extent of inconsistency in full networks of interventions or the factors that control its statistical detection.
  
  
  METHODS
  In this paper we assess the prevalence of inconsistency from data of 40 published networks of interventions involving 303 loops of evidence. Inconsistency is evaluated in each loop by contrasting direct and indirect estimates and by employing an omnibus test of consistency for the entire network. We explore whether different effect measures for dichotomous outcomes are associated with differences in inconsistency, and evaluate whether different ways to estimate heterogeneity affect the magnitude and detection of inconsistency.
  
  
  RESULTS
  Inconsistency was detected in from 2% to 9% of the tested loops, depending on the effect measure and heterogeneity estimation method. Loops that included comparisons informed by a single study were more likely to show inconsistency. About one-eighth of the networks were found to be inconsistent. The proportions of inconsistent loops do not materially change when different effect measures are used. Important heterogeneity or the overestimation of heterogeneity was associated with a small decrease in the prevalence of statistical inconsistency.
  
  
  CONCLUSIONS
  The study suggests that changing the effect measure might improve statistical consistency, and that an analysis of sensitivity to the assumptions and an estimator of heterogeneity might be needed before reaching a conclusion about the absence of statistical inconsistency, particularly in networks with few studies.
authors.affiliation: Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece.
authors.forename: Areti Angeliki
authors.initials: AA
authors.lastname: Veroniki
authors.forename: Haris S
authors.initials: HS
authors.lastname: Vasiliadis
authors.forename: Julian P T
authors.initials: JP
authors.lastname: Higgins
authors.forename: Georgia
authors.initials: G
authors.lastname: Salanti
mesh_terms: D006801:Humans; D015201:Meta-Analysis as Topic; D015233:Models, Statistical; D016571:Neural Networks, Computer; D016017:Odds Ratio; D015203:Reproducibility of Results; D012107:Research Design; D016896:Treatment Outcome