Diabetes healthcare quality report cards: how accurate are the grades?
OBJECTIVE
To evaluate the accuracy and precision of random sampling in identifying healthcare system outliers in diabetes performance measures.


METHODS
Cross-sectional analysis of 79 Veterans Health Administration facilities serving 250 317 patients with diabetes mellitus between October 1, 1999, and September 30, 2000.


METHODS
Primary outcome measures were poor glycosylated hemoglobin (A1C) control and good low-density lipoprotein cholesterol (LDL-C) and blood pressure (BP) control. Facility performance for each measure was calculated using 150 separate random samples and was compared with results using the bootstrap method as the criterion standard for determining outlier status (defined as a >/=5% difference from the mean, within the 10th or 90th percentile, or >/=2 SDs from the mean).


RESULTS
The study population was largely male (97.4%), with 54.0% of subjects being 65 years or older. The facility-level mean performances were 22.8% for poor A1C control, 53.1% for good LDL-C control, and 55.3% for good BP control. Comparing the random sampling method with the bootstrap method, the sensitivity ranged between 0.64 and 0.83 for the 3 outcome measures, positive predictive values ranged between 0.55 and 0.88, and specificity and negative predictive values ranged between 0.88 and 0.99.


CONCLUSIONS
The specificity and negative predictive value of the random sampling method in identifying nonoutliers in performance were generally high, while its sensitivity and positive predictive value were moderate. The use of random sampling to determine performance for individual outcome measures may be most appropriate for internal quality improvement rather than for public reporting.
Pogach
Leonard
L
Center for Healthcare Knowledge Management, VA New Jersey Health Care System, East Orange, USA. leonard.pogach@med.va.gov.
Xie
Minge
M
Shentue
Yue
Y
Tseng
Chin-Lin
CL
Maney
Miriam
M
Rajan
Mangala
M
Tiwari
Anjali
A
Kolassa
John
J
Helmer
Drew
D
Crystal
Stephen
S
Safford
Monika
M
D000368:Aged; D001794:Blood Pressure; D008078:Cholesterol, LDL / Q000097:blood; D003430:Cross-Sectional Studies; D003920:Diabetes Mellitus / Q000175:diagnosis / Q000628:therapy*; D005260:Female; D006442:Glycated Hemoglobin / Q000032:analysis; D006787:Hospitals, Veterans / Q000592:standards*; D006801:Humans; D033181:Information Dissemination; D008297:Male; D008875:Middle Aged; D011237:Predictive Value of Tests; D019984:Quality Indicators, Health Care; D012494:Sampling Studies; D014481:United States; D014493:United States Department of Veterans Affairs
D008078:Cholesterol, LDL; D006442:Glycated Hemoglobin