title: Measuring performance in chest radiography.
abstract: OBJECTIVE
  To use a standardized set of chest radiographs to quantify interobserver differences and to provide a basis for comparing the diagnostic performance of physicians.
  
  
  METHODS
  A standardized set of 60 chest radiographs was presented to 162 study participants. Each participant reviewed the radiographs and recorded his or her diagnostic impression by using a fixed five-point scale. These response data were used to generate receiver operating characteristic curves and to establish performance benchmarks. The variations in performance were tested for statistical significance.
  
  
  RESULTS
  Significant interobserver variability was identified during these assessments. The composite group of board-certified radiologists demonstrated performance superior to that of the radiology residents and nonradiologist physicians.
  
  
  CONCLUSIONS
  By using a receiver operating characteristic approach and a standardized set of chest radiographs, observer accuracy and variability are easily quantified. This approach provides a basis for comparing the diagnostic performance of physicians. When value is measured as a diminution in uncertainty, board-certified radiologists contribute substantial value to the diagnostic imaging system.
authors.affiliation: Department of Radiology, Michigan State University, 164 Radiology Bldg, East Lansing, MI 48824, USA. ejp@rad.msu.edu
authors.forename: E J
authors.initials: EJ
authors.lastname: Potchen
authors.forename: T G
authors.initials: TG
authors.lastname: Cooper
authors.forename: A E
authors.initials: AE
authors.lastname: Sierra
authors.forename: G R
authors.initials: GR
authors.lastname: Aben
authors.forename: M J
authors.initials: MJ
authors.lastname: Potchen
authors.forename: M G
authors.initials: MG
authors.lastname: Potter
authors.forename: J E
authors.initials: JE
authors.lastname: Siebert
mesh_terms: D002568:Certification; D002983:Clinical Competence; D006801:Humans; D007396:Internship and Residency; D015588:Observer Variation; D010820:Physicians; D012372:ROC Curve; D013902:Radiography, Thoracic; D011871:Radiology / Q000193:education; D013896:Thoracic Diseases / Q000000981:diagnostic imaging*