title: Observer performance studies: detection of single versus multiple abnormalities of the chest.
abstract: OBJECTIVE
  We used receiver operating characteristic (ROC) analysis to compare two methods of evaluating observer performance in detecting an abnormality on chest radiographs. In the first method, the abnormality in question, rib fracture, was one of five investigated, and it was the only one of interest in the second.
  
  
  METHODS
  Eight experienced observers viewed 117 posteroanterior chest radiographs in two interpretation modes. Fifty-four of these images depicted rib fractures that had been rated as subtle for detection. The likelihood of the presence of a rib fracture was rated as one of five abnormalities in question in one mode and the sole abnormality of interest in the other mode.
  
  
  RESULTS
  Six of the observers performed better during the single-abnormality mode, one performed equally well in both modes, and one performed better during the multiple-abnormality mode. The average area under the ROC curves (A(z)) was 0.73 +/- 0.07 for the multiple-abnormality mode and 0.80 +/- 0.04 for the single-abnormality mode. The results were significantly different (p < 0.05).
  
  
  CONCLUSIONS
  Study methodology can significantly affect the results in ROC studies, particularly for abnormalities that may not be perceived as primary or important. The order in which abnormalities appear on a checklist report form may be important.
authors.affiliation: Department of Radiology, University of Pittsburgh, 200 Lothrop St., Pittsburgh, PA 15213-2582, USA.
authors.forename: Carl R
authors.initials: CR
authors.lastname: Fuhrman
authors.forename: Cynthia A
authors.initials: CA
authors.lastname: Britton
authors.forename: Thomas
authors.initials: T
authors.lastname: Bender
authors.forename: Jules H
authors.initials: JH
authors.lastname: Sumkin
authors.forename: Manuel L
authors.initials: ML
authors.lastname: Brown
authors.forename: J Michael
authors.initials: JM
authors.lastname: Holbert
authors.forename: Thomas S
authors.initials: TS
authors.lastname: Chang
authors.forename: Howard E
authors.initials: HE
authors.lastname: Rockette
authors.forename: David
authors.initials: D
authors.lastname: Gur
mesh_terms: D019540:Area Under Curve; D006801:Humans; D008171:Lung Diseases / Q000150:complications / Q000000981:diagnostic imaging*; D015588:Observer Variation; D012372:ROC Curve; D013902:Radiography, Thoracic; D012253:Rib Fractures / Q000150:complications / Q000000981:diagnostic imaging*