title: Setting a fair performance standard for physicians' quality of patient care.
abstract: BACKGROUND
  Assessing physicians' clinical performance using statistically sound, evidence-based measures is challenging. Little research has focused on methodological approaches to setting performance standards to which physicians are being held accountable.
  
  
  OBJECTIVE
  Determine if a rigorous approach for setting an objective, credible standard of minimally-acceptable performance could be used for practicing physicians caring for diabetic patients.
  
  
  METHODS
  Retrospective cohort study.
  
  
  METHODS
  Nine hundred and fifty-seven physicians from the United States with time-limited certification in internal medicine or a subspecialty.
  
  
  METHODS
  The ABIM Diabetes Practice Improvement Module was used to collect data on ten clinical and two patient experience measures. A panel of eight internists/subspecialists representing essential perspectives of clinical practice applied an adaptation of the Angoff method to judge how physicians who provide minimally-acceptable care would perform on individual measures to establish performance thresholds. Panelists then rated each measure's relative importance and the Dunn-Rankin method was applied to establish scoring weights for the composite measure. Physician characteristics were used to support the standard-setting outcome.
  
  
  RESULTS
  Physicians abstracted 20,131 patient charts and 18,974 patient surveys were completed. The panel established reasonable performance thresholds and importance weights, yielding a standard of 48.51 (out of 100 possible points) on the composite measure with high classification accuracy (0.98). The 38 (4%) outlier physicians who did not meet the standard had lower ratings of overall clinical competence and professional behavior/attitude from former residency program directors (p = 0.01 and p = 0.006, respectively), lower Internal Medicine certification and maintenance of certification examination scores (p = 0.005 and p < 0.001, respectively), and primarily worked as solo practitioners (p = 0.02).
  
  
  CONCLUSIONS
  The standard-setting method yielded a credible, defensible performance standard for diabetes care based on informed judgment that resulted in a reasonable, reproducible outcome. Our method represents one approach to identifying outlier physicians for intervention to protect patients.
authors.affiliation: American Board of Internal Medicine, Philadelphia, PA 19106, USA. bhess@abim.org
authors.forename: Brian J
authors.initials: BJ
authors.lastname: Hess
authors.forename: Weifeng
authors.initials: W
authors.lastname: Weng
authors.forename: Lorna A
authors.initials: LA
authors.lastname: Lynn
authors.forename: Eric S
authors.initials: ES
authors.lastname: Holmboe
authors.forename: Rebecca S
authors.initials: RS
authors.lastname: Lipner
mesh_terms: D000293:Adolescent; D000328:Adult; D000368:Aged; D002983:Clinical Competence / Q000592:standards*; D015331:Cohort Studies; D003920:Diabetes Mellitus / Q000175:diagnosis / Q000628:therapy; D004650:Employee Performance Appraisal / Q000592:standards*; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D005791:Patient Care / Q000379:methods / Q000592:standards*; D010820:Physicians / Q000592:standards*; D011446:Prospective Studies; D011787:Quality of Health Care / Q000592:standards*; D012189:Retrospective Studies; D055815:Young Adult