title: GRADE guidelines: 7. Rating the quality of evidence--inconsistency.
abstract: This article deals with inconsistency of relative (rather than absolute) treatment effects in binary/dichotomous outcomes. A body of evidence is not rated up in quality if studies yield consistent results, but may be rated down in quality if inconsistent. Criteria for evaluating consistency include similarity of point estimates, extent of overlap of confidence intervals, and statistical criteria including tests of heterogeneity and I(2). To explore heterogeneity, systematic review authors should generate and test a small number of a priori hypotheses related to patients, interventions, outcomes, and methodology. When inconsistency is large and unexplained, rating down quality for inconsistency is appropriate, particularly if some studies suggest substantial benefit, and others no effect or harm (rather than only large vs. small effects). Apparent subgroup effects may be spurious. Credibility is increased if subgroup effects are based on a small number of a priori hypotheses with a specified direction; subgroup comparisons come from within rather than between studies; tests of interaction generate low P-values; and have a biological rationale.
authors.affiliation: Department of Clinical Epidemiology and Biostatistics, McMaster University, 1200 Main Street, West Hamilton, Ontario L8N 3Z5, Canada. guyatt@mcmaster.ca
authors.forename: Gordon H
authors.initials: GH
authors.lastname: Guyatt
authors.forename: Andrew D
authors.initials: AD
authors.lastname: Oxman
authors.forename: Regina
authors.initials: R
authors.lastname: Kunz
authors.forename: James
authors.initials: J
authors.lastname: Woodcock
authors.forename: Jan
authors.initials: J
authors.lastname: Brozek
authors.forename: Mark
authors.initials: M
authors.lastname: Helfand
authors.forename: Pablo
authors.initials: P
authors.lastname: Alonso-Coello
authors.forename: Paul
authors.initials: P
authors.lastname: Glasziou
authors.forename: Roman
authors.initials: R
authors.lastname: Jaeschke
authors.forename: Elie A
authors.initials: EA
authors.lastname: Akl
authors.forename: Susan
authors.initials: S
authors.lastname: Norris
authors.forename: Gunn
authors.initials: G
authors.lastname: Vist
authors.forename: Philipp
authors.initials: P
authors.lastname: Dahm
authors.forename: Vijay K
authors.initials: VK
authors.lastname: Shukla
authors.forename: Julian
authors.initials: J
authors.lastname: Higgins
authors.forename: Yngve
authors.initials: Y
authors.lastname: Falck-Ytter
authors.forename: Holger J
authors.initials: HJ
authors.lastname: Schünemann
mesh_terms: D016001:Confidence Intervals; D019317:Evidence-Based Medicine / Q000592:standards*; D006801:Humans; D015588:Observer Variation; D017410:Practice Guidelines as Topic; D016032:Randomized Controlled Trials as Topic / Q000592:standards; D012107:Research Design; D018401:Sample Size