Measuring performance in chest radiography.
OBJECTIVE
To use a standardized set of chest radiographs to quantify interobserver differences and to provide a basis for comparing the diagnostic performance of physicians.


METHODS
A standardized set of 60 chest radiographs was presented to 162 study participants. Each participant reviewed the radiographs and recorded his or her diagnostic impression by using a fixed five-point scale. These response data were used to generate receiver operating characteristic curves and to establish performance benchmarks. The variations in performance were tested for statistical significance.


RESULTS
Significant interobserver variability was identified during these assessments. The composite group of board-certified radiologists demonstrated performance superior to that of the radiology residents and nonradiologist physicians.


CONCLUSIONS
By using a receiver operating characteristic approach and a standardized set of chest radiographs, observer accuracy and variability are easily quantified. This approach provides a basis for comparing the diagnostic performance of physicians. When value is measured as a diminution in uncertainty, board-certified radiologists contribute substantial value to the diagnostic imaging system.
Potchen
E J
EJ
Department of Radiology, Michigan State University, 164 Radiology Bldg, East Lansing, MI 48824, USA. ejp@rad.msu.edu
Cooper
T G
TG
Sierra
A E
AE
Aben
G R
GR
Potchen
M J
MJ
Potter
M G
MG
Siebert
J E
JE
D002568:Certification; D002983:Clinical Competence; D006801:Humans; D007396:Internship and Residency; D015588:Observer Variation; D010820:Physicians; D012372:ROC Curve; D013902:Radiography, Thoracic; D011871:Radiology / Q000193:education; D013896:Thoracic Diseases / Q000000981:diagnostic imaging*