Meaningful variation in performance: a systematic literature review.
BACKGROUND
Recommendations for directing quality improvement initiatives at particular levels (eg, patients, physicians, provider groups) have been made on the basis of empirical components of variance analyses of performance.


OBJECTIVE
To review the literature on use of multilevel analyses of variability in quality.


METHODS
Systematic literature review of English-language articles (n = 39) examining variability and reliability of performance measures in Medline using PubMed (1949-November 2008).


RESULTS
Variation was most commonly assessed at facility (eg, hospital, medical center) (n = 19) and physician (n = 18) levels; most articles reported variability as the proportion of total variation attributable to given levels (n = 22). Proportions of variability explained by aggregated levels were generally low (eg, <19% for physicians), and numerous authors concluded that the proportion of variability at a specific level did not justify targeting quality interventions to that level. Few articles based their recommendations on absolute differences among physicians, hospitals, or other levels. Seven of 12 articles that assessed reliability found that reliability was poor at the physician or hospital level due to low proportional variability and small sample sizes per unit, and cautioned that public reporting or incentives based on these measures may be inappropriate.


CONCLUSIONS
The proportion of variability at levels higher than patients is often found to be "low." Although low proportional variability may lead to poor measurement reliability, a number of authors further suggested that it also indicates a lack of potential for quality improvement. Few studies provided additional information to help determine whether variation was, nevertheless, clinically meaningful.
Fung
Vicki
V
Division of Research, Kaiser Permanente Medical Care Program, Oakland, CA, USA. vicki.fung@kp.org
Schmittdiel
Julie A
JA
Fireman
Bruce
B
Meer
Aabed
A
Thomas
Sean
S
Smider
Nancy
N
Hsu
John
J
Selby
Joseph V
JV
D002983:Clinical Competence; D006801:Humans; D010043:Outcome and Process Assessment, Health Care; D010818:Practice Patterns, Physicians'; D011785:Quality Assurance, Health Care; D019984:Quality Indicators, Health Care; D015203:Reproducibility of Results