Alternative explanations for poor report card performance.
BACKGROUND
Many managed care organizations grade physician groups with "report cards" developed from administrative data sets and chart reviews.


OBJECTIVE
To investigate the accuracy of five report cards on a single group practice.


METHODS
Determination of report card accuracy by using the practice capitation list and a review of the patients' medical records.


METHODS
Academic practice in Philadelphia, Pennsylvania (19 physicians), evaluated with five report cards by two capitated health plans between 1994 and 1997.


RESULTS
Four major problems were uncovered. First, four of the five report cards included patients who were enrolled in our practice for only a portion of the reporting year (for the four report cards, the proportion of partial-year enrollees was 8%, 15%, 23%, and 100%). Second, there was a considerable number of false-positive diagnoses in the administrative algorithms. Eight of the 61 patients labeled with hypertension did not have this condition (error rate, 14%). Other error rates were 44% for coronary artery disease, 50% for congestive heart failure, 33% for atrial fibrillation, and 0% for diabetes. Third, the administrative data often failed to capture laboratory data. Laboratory performance measures for patients with diabetes (hemoglobin A1c and cholesterol measurement and screening for microalbuminuria) were 3 to 10 times higher when assessed by chart review. Finally, the uniformly small sample sizes used in the report cards make the estimates of performance imprecise. No report card reported 95% CIs.


CONCLUSIONS
Five report cards on a group practice contained methodologic problems that led to systematic underestimation of the practice's performance. Larger surveys are needed to determine the accuracy of report cards in current use.
Sorokin
R
R
Jefferson Medical College, Philadelphia, Penn., USA. rachel.sorokin@mail.tju.edu
D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D005260:Female; D006123:Group Practice / Q000592:standards; D006801:Humans; D008297:Male; D008329:Managed Care Programs / Q000592:standards*; D008485:Medical Audit / Q000706:statistics & numerical data*; D008875:Middle Aged; D019982:Organizational Case Studies; D015143:Philadelphia; D011320:Primary Health Care / Q000592:standards*; D015203:Reproducibility of Results