title: Diagnostic mammography: identifying minimally acceptable interpretive performance criteria.
abstract: OBJECTIVE
  To develop criteria to identify thresholds for the minimally acceptable performance of physicians interpreting diagnostic mammography studies.
  
  
  METHODS
  In an institutional review board-approved HIPAA-compliant study, an Angoff approach was used to set criteria for identifying minimally acceptable interpretive performance for both workup after abnormal screening examinations and workup of a breast lump. Normative data from the Breast Cancer Surveillance Consortium (BCSC) was used to help the expert radiologist identify the impact of cut points. Simulations, also using data from the BCSC, were used to estimate the expected clinical impact from the recommended performance thresholds.
  
  
  RESULTS
  Final cut points for workup of abnormal screening examinations were as follows: sensitivity, less than 80%; specificity, less than 80% or greater than 95%; abnormal interpretation rate, less than 8% or greater than 25%; positive predictive value (PPV) of biopsy recommendation (PPV2), less than 15% or greater than 40%; PPV of biopsy performed (PPV3), less than 20% or greater than 45%; and cancer diagnosis rate, less than 20 per 1000 interpretations. Final cut points for workup of a breast lump were as follows: sensitivity, less than 85%; specificity, less than 83% or greater than 95%; abnormal interpretation rate, less than 10% or greater than 25%; PPV2, less than 25% or greater than 50%; PPV3, less than 30% or greater than 55%; and cancer diagnosis rate, less than 40 per 1000 interpretations. If underperforming physicians moved into the acceptable range after remedial training, the expected result would be (a) diagnosis of an additional 86 cancers per 100,000 women undergoing workup after screening examinations, with a reduction in the number of false-positive examinations by 1067 per 100,000 women undergoing this workup, and (b) diagnosis of an additional 335 cancers per 100,000 women undergoing workup of a breast lump, with a reduction in the number of false-positive examinations by 634 per 100,000 women undergoing this workup.
  
  
  CONCLUSIONS
  Interpreting physicians who fall outside one or more of the identified cut points should be reviewed in the context of an overall assessment of all their performance measures and their specific practice setting to determine if remedial training is indicated.
authors.affiliation: Departments of Family Medicine and Public Health and Preventive Medicine, Oregon Health and Science University, 3181 SW Sam Jackson Park Rd, Portland, OR 97239-3098, USA. carneyp@ohsu.edu
authors.forename: Patricia A
authors.initials: PA
authors.lastname: Carney
authors.forename: Jay
authors.initials: J
authors.lastname: Parikh
authors.forename: Edward A
authors.initials: EA
authors.lastname: Sickles
authors.forename: Stephen A
authors.initials: SA
authors.lastname: Feig
authors.forename: Barbara
authors.initials: B
authors.lastname: Monsees
authors.forename: Lawrence W
authors.initials: LW
authors.lastname: Bassett
authors.forename: Robert A
authors.initials: RA
authors.lastname: Smith
authors.forename: Robert
authors.initials: R
authors.lastname: Rosenberg
authors.forename: Laura
authors.initials: L
authors.lastname: Ichikawa
authors.forename: James
authors.initials: J
authors.lastname: Wallace
authors.forename: Khai
authors.initials: K
authors.lastname: Tran
authors.forename: Diana L
authors.initials: DL
authors.lastname: Miglioretti
mesh_terms: D019985:Benchmarking; D001706:Biopsy / Q000592:standards; D001943:Breast Neoplasms / Q000000981:diagnostic imaging*; D002983:Clinical Competence / Q000592:standards; D003627:Data Interpretation, Statistical; D005260:Female; D006801:Humans; D008327:Mammography / Q000592:standards*; D008403:Mass Screening / Q000592:standards; D012680:Sensitivity and Specificity