Effect of CAD on radiologists' detection of lung nodules on thoracic CT scans: analysis of an observer performance study by nodule size.
OBJECTIVE
To retrospectively investigate the effect of a computer-aided detection (CAD) system on radiologists' performance for detecting small pulmonary nodules in computed tomography (CT) examinations, with a panel of expert radiologists serving as the reference standard.


METHODS
Institutional review board approval was obtained. Our dataset contained 52 CT examinations collected by the Lung Image Database Consortium, and 33 from our institution. All CTs were read by multiple expert thoracic radiologists to identify the reference standard for detection. Six other thoracic radiologists read the CT examinations first without and then with CAD. Performance was evaluated using free-response receiver operating characteristics (FROC) and the jackknife FROC analysis methods (JAFROC) for nodules above different diameter thresholds.


RESULTS
A total of 241 nodules, ranging in size from 3.0 to 18.6 mm (mean, 5.3 mm) were identified as the reference standard. At diameter thresholds of 3, 4, 5, and 6 mm, the CAD system had a sensitivity of 54%, 64%, 68%, and 76%, respectively, with an average of 5.6 false positives (FPs) per scan. Without CAD, the average figures of merit (FOMs) for the six radiologists, obtained from JAFROC analysis, were 0.661, 0.729, 0.793, and 0.838 for the same nodule diameter thresholds, respectively. With CAD, the corresponding average FOMs improved to 0.705, 0.763, 0.810, and 0.862, respectively. The improvement achieved statistical significance for nodules at the 3 and 4 mm thresholds (P = .002 and .020, respectively), and did not achieve significance at 5 and 6 mm (P = .18 and .13, respectively). At a nodule diameter threshold of 3 mm, the radiologists' average sensitivity and FP rate were 0.56 and 0.67, respectively, without CAD, and 0.67 and 0.78 with CAD.


CONCLUSIONS
CAD improves thoracic radiologists' performance for detecting pulmonary nodules smaller than 5 mm on CT examinations, which are often overlooked by visual inspection alone.
Sahiner
Berkman
B
Department of Radiology, The University of Michigan, MIB C480A, 1500 East Medical Center Drive, Ann Arbor, MI 48109, USA. berki@umich.edu
Chan
Heang-Ping
HP
Hadjiiski
Lubomir M
LM
Cascade
Philip N
PN
Kazerooni
Ella A
EA
Chughtai
Aamer R
AR
Poopat
Chad
C
Song
Thomas
T
Frank
Luba
L
Stojanovska
Jadranka
J
Attili
Anil
A
D000368:Aged; D005260:Female; D006801:Humans; D008175:Lung Neoplasms / Q000000981:diagnostic imaging*; D008297:Male; D008875:Middle Aged; D015588:Observer Variation; D010363:Pattern Recognition, Automated / Q000379:methods*; D011856:Radiographic Image Enhancement / Q000379:methods; D011857:Radiographic Image Interpretation, Computer-Assisted / Q000379:methods*; D013902:Radiography, Thoracic / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D003074:Solitary Pulmonary Nodule / Q000000981:diagnostic imaging*; D014057:Tomography, X-Ray Computed / Q000379:methods*