title: Alternative explanations for poor report card performance.
abstract: BACKGROUND
  Many managed care organizations grade physician groups with "report cards" developed from administrative data sets and chart reviews.
  
  
  OBJECTIVE
  To investigate the accuracy of five report cards on a single group practice.
  
  
  METHODS
  Determination of report card accuracy by using the practice capitation list and a review of the patients' medical records.
  
  
  METHODS
  Academic practice in Philadelphia, Pennsylvania (19 physicians), evaluated with five report cards by two capitated health plans between 1994 and 1997.
  
  
  RESULTS
  Four major problems were uncovered. First, four of the five report cards included patients who were enrolled in our practice for only a portion of the reporting year (for the four report cards, the proportion of partial-year enrollees was 8%, 15%, 23%, and 100%). Second, there was a considerable number of false-positive diagnoses in the administrative algorithms. Eight of the 61 patients labeled with hypertension did not have this condition (error rate, 14%). Other error rates were 44% for coronary artery disease, 50% for congestive heart failure, 33% for atrial fibrillation, and 0% for diabetes. Third, the administrative data often failed to capture laboratory data. Laboratory performance measures for patients with diabetes (hemoglobin A1c and cholesterol measurement and screening for microalbuminuria) were 3 to 10 times higher when assessed by chart review. Finally, the uniformly small sample sizes used in the report cards make the estimates of performance imprecise. No report card reported 95% CIs.
  
  
  CONCLUSIONS
  Five report cards on a group practice contained methodologic problems that led to systematic underestimation of the practice's performance. Larger surveys are needed to determine the accuracy of report cards in current use.
authors.affiliation: Jefferson Medical College, Philadelphia, Penn., USA. rachel.sorokin@mail.tju.edu
authors.forename: R
authors.initials: R
authors.lastname: Sorokin
mesh_terms: D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D005260:Female; D006123:Group Practice / Q000592:standards; D006801:Humans; D008297:Male; D008329:Managed Care Programs / Q000592:standards*; D008485:Medical Audit / Q000706:statistics & numerical data*; D008875:Middle Aged; D019982:Organizational Case Studies; D015143:Philadelphia; D011320:Primary Health Care / Q000592:standards*; D015203:Reproducibility of Results