title: Effect of CAD on radiologists' detection of lung nodules on thoracic CT scans: analysis of an observer performance study by nodule size.
abstract: OBJECTIVE
  To retrospectively investigate the effect of a computer-aided detection (CAD) system on radiologists' performance for detecting small pulmonary nodules in computed tomography (CT) examinations, with a panel of expert radiologists serving as the reference standard.
  
  
  METHODS
  Institutional review board approval was obtained. Our dataset contained 52 CT examinations collected by the Lung Image Database Consortium, and 33 from our institution. All CTs were read by multiple expert thoracic radiologists to identify the reference standard for detection. Six other thoracic radiologists read the CT examinations first without and then with CAD. Performance was evaluated using free-response receiver operating characteristics (FROC) and the jackknife FROC analysis methods (JAFROC) for nodules above different diameter thresholds.
  
  
  RESULTS
  A total of 241 nodules, ranging in size from 3.0 to 18.6 mm (mean, 5.3 mm) were identified as the reference standard. At diameter thresholds of 3, 4, 5, and 6 mm, the CAD system had a sensitivity of 54%, 64%, 68%, and 76%, respectively, with an average of 5.6 false positives (FPs) per scan. Without CAD, the average figures of merit (FOMs) for the six radiologists, obtained from JAFROC analysis, were 0.661, 0.729, 0.793, and 0.838 for the same nodule diameter thresholds, respectively. With CAD, the corresponding average FOMs improved to 0.705, 0.763, 0.810, and 0.862, respectively. The improvement achieved statistical significance for nodules at the 3 and 4 mm thresholds (P = .002 and .020, respectively), and did not achieve significance at 5 and 6 mm (P = .18 and .13, respectively). At a nodule diameter threshold of 3 mm, the radiologists' average sensitivity and FP rate were 0.56 and 0.67, respectively, without CAD, and 0.67 and 0.78 with CAD.
  
  
  CONCLUSIONS
  CAD improves thoracic radiologists' performance for detecting pulmonary nodules smaller than 5 mm on CT examinations, which are often overlooked by visual inspection alone.
authors.affiliation: Department of Radiology, The University of Michigan, MIB C480A, 1500 East Medical Center Drive, Ann Arbor, MI 48109, USA. berki@umich.edu
authors.forename: Berkman
authors.initials: B
authors.lastname: Sahiner
authors.forename: Heang-Ping
authors.initials: HP
authors.lastname: Chan
authors.forename: Lubomir M
authors.initials: LM
authors.lastname: Hadjiiski
authors.forename: Philip N
authors.initials: PN
authors.lastname: Cascade
authors.forename: Ella A
authors.initials: EA
authors.lastname: Kazerooni
authors.forename: Aamer R
authors.initials: AR
authors.lastname: Chughtai
authors.forename: Chad
authors.initials: C
authors.lastname: Poopat
authors.forename: Thomas
authors.initials: T
authors.lastname: Song
authors.forename: Luba
authors.initials: L
authors.lastname: Frank
authors.forename: Jadranka
authors.initials: J
authors.lastname: Stojanovska
authors.forename: Anil
authors.initials: A
authors.lastname: Attili
mesh_terms: D000368:Aged; D005260:Female; D006801:Humans; D008175:Lung Neoplasms / Q000000981:diagnostic imaging*; D008297:Male; D008875:Middle Aged; D015588:Observer Variation; D010363:Pattern Recognition, Automated / Q000379:methods*; D011856:Radiographic Image Enhancement / Q000379:methods; D011857:Radiographic Image Interpretation, Computer-Assisted / Q000379:methods*; D013902:Radiography, Thoracic / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D003074:Solitary Pulmonary Nodule / Q000000981:diagnostic imaging*; D014057:Tomography, X-Ray Computed / Q000379:methods*