title: Meaningful variation in performance: a systematic literature review.
abstract: BACKGROUND
  Recommendations for directing quality improvement initiatives at particular levels (eg, patients, physicians, provider groups) have been made on the basis of empirical components of variance analyses of performance.
  
  
  OBJECTIVE
  To review the literature on use of multilevel analyses of variability in quality.
  
  
  METHODS
  Systematic literature review of English-language articles (n = 39) examining variability and reliability of performance measures in Medline using PubMed (1949-November 2008).
  
  
  RESULTS
  Variation was most commonly assessed at facility (eg, hospital, medical center) (n = 19) and physician (n = 18) levels; most articles reported variability as the proportion of total variation attributable to given levels (n = 22). Proportions of variability explained by aggregated levels were generally low (eg, <19% for physicians), and numerous authors concluded that the proportion of variability at a specific level did not justify targeting quality interventions to that level. Few articles based their recommendations on absolute differences among physicians, hospitals, or other levels. Seven of 12 articles that assessed reliability found that reliability was poor at the physician or hospital level due to low proportional variability and small sample sizes per unit, and cautioned that public reporting or incentives based on these measures may be inappropriate.
  
  
  CONCLUSIONS
  The proportion of variability at levels higher than patients is often found to be "low." Although low proportional variability may lead to poor measurement reliability, a number of authors further suggested that it also indicates a lack of potential for quality improvement. Few studies provided additional information to help determine whether variation was, nevertheless, clinically meaningful.
authors.affiliation: Division of Research, Kaiser Permanente Medical Care Program, Oakland, CA, USA. vicki.fung@kp.org
authors.forename: Vicki
authors.initials: V
authors.lastname: Fung
authors.forename: Julie A
authors.initials: JA
authors.lastname: Schmittdiel
authors.forename: Bruce
authors.initials: B
authors.lastname: Fireman
authors.forename: Aabed
authors.initials: A
authors.lastname: Meer
authors.forename: Sean
authors.initials: S
authors.lastname: Thomas
authors.forename: Nancy
authors.initials: N
authors.lastname: Smider
authors.forename: John
authors.initials: J
authors.lastname: Hsu
authors.forename: Joseph V
authors.initials: JV
authors.lastname: Selby
mesh_terms: D002983:Clinical Competence; D006801:Humans; D010043:Outcome and Process Assessment, Health Care; D010818:Practice Patterns, Physicians'; D011785:Quality Assurance, Health Care; D019984:Quality Indicators, Health Care; D015203:Reproducibility of Results