Investigation of reading mode and relative sensitivity as factors that influence reader performance when using computer-aided detection software.
OBJECTIVE
The aim of this study was to investigate the effects of relative sensitivity (reader without computer-aided detection [CAD] vs stand-alone CAD) and reading mode on reader performance when using CAD software.


METHODS
Two sets of 100 images (low-contrast and high-contrast sets) were created by adding low-contrast or high-contrast simulated masses to random locations in 100 normal mammograms. This produced a relative sensitivity, substantially less for the low-contrast set and similar for the high-contrast set. Seven readers reviewed every image in each set and specified location and probability scores using three reading modes (without CAD, second read with CAD, and concurrent read with CAD). Reader detection accuracy was analyzed using areas under free-response receiver operating characteristic curves, sensitivity, and the number of false-positive findings per image.


RESULTS
For the low-contrast set, average differences in areas under free-response receiver operating characteristic curves, sensitivity, and false-positive findings per image without CAD were 0.02, 0.12, and 0.11, respectively, compared to second read and 0.05, 0.17, and 0.09 (not statistically significant), respectively, compared to concurrent read. For the high-contrast set, average differences were 0.002 (not statistically significant), 0.04, and 0.05, respectively, compared to second read and -0.004 (not statistically significant), 0.04, and 0.08 (not statistically significant), respectively, compared to concurrent read (all differences were statistically significant except as noted). Differences were greater in the low-contrast set than the high-contrast set. Differences between second read and concurrent read were not significant.


CONCLUSIONS
Relative sensitivity is a critical factor that determines incremental improvement in reader performance when using CAD and appears to be more important than reading mode. Relative sensitivity may determine the clinical usefulness of CAD in different clinical applications and for different types of users.
Paquerault
Sophie
S
Center for Devices and Radiological Health/National Insitute of Biomedical Imaging and Bioengineering, Joint Laboratory for Assessment of Medical Imaging Systems, US Food and Drug Administration, Silver Spring, MD 20993-0002, USA. sophie.paquerault@fda.hhs.gov
Samuelson
Frank W
FW
Petrick
Nicholas
N
Myers
Kyle J
KJ
Smith
Robert C
RC
D000465:Algorithms; D016477:Artifacts; D001185:Artificial Intelligence; D001943:Breast Neoplasms / Q000000981:diagnostic imaging*; D005260:Female; D006801:Humans; D008327:Mammography / Q000379:methods*; D015588:Observer Variation; D010363:Pattern Recognition, Automated / Q000379:methods*; D011856:Radiographic Image Enhancement / Q000379:methods; D011857:Radiographic Image Interpretation, Computer-Assisted / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D012984:Software; D012986:Software Validation