title: Investigation of reading mode and relative sensitivity as factors that influence reader performance when using computer-aided detection software.
abstract: OBJECTIVE
  The aim of this study was to investigate the effects of relative sensitivity (reader without computer-aided detection [CAD] vs stand-alone CAD) and reading mode on reader performance when using CAD software.
  
  
  METHODS
  Two sets of 100 images (low-contrast and high-contrast sets) were created by adding low-contrast or high-contrast simulated masses to random locations in 100 normal mammograms. This produced a relative sensitivity, substantially less for the low-contrast set and similar for the high-contrast set. Seven readers reviewed every image in each set and specified location and probability scores using three reading modes (without CAD, second read with CAD, and concurrent read with CAD). Reader detection accuracy was analyzed using areas under free-response receiver operating characteristic curves, sensitivity, and the number of false-positive findings per image.
  
  
  RESULTS
  For the low-contrast set, average differences in areas under free-response receiver operating characteristic curves, sensitivity, and false-positive findings per image without CAD were 0.02, 0.12, and 0.11, respectively, compared to second read and 0.05, 0.17, and 0.09 (not statistically significant), respectively, compared to concurrent read. For the high-contrast set, average differences were 0.002 (not statistically significant), 0.04, and 0.05, respectively, compared to second read and -0.004 (not statistically significant), 0.04, and 0.08 (not statistically significant), respectively, compared to concurrent read (all differences were statistically significant except as noted). Differences were greater in the low-contrast set than the high-contrast set. Differences between second read and concurrent read were not significant.
  
  
  CONCLUSIONS
  Relative sensitivity is a critical factor that determines incremental improvement in reader performance when using CAD and appears to be more important than reading mode. Relative sensitivity may determine the clinical usefulness of CAD in different clinical applications and for different types of users.
authors.affiliation: Center for Devices and Radiological Health/National Insitute of Biomedical Imaging and Bioengineering, Joint Laboratory for Assessment of Medical Imaging Systems, US Food and Drug Administration, Silver Spring, MD 20993-0002, USA. sophie.paquerault@fda.hhs.gov
authors.forename: Sophie
authors.initials: S
authors.lastname: Paquerault
authors.forename: Frank W
authors.initials: FW
authors.lastname: Samuelson
authors.forename: Nicholas
authors.initials: N
authors.lastname: Petrick
authors.forename: Kyle J
authors.initials: KJ
authors.lastname: Myers
authors.forename: Robert C
authors.initials: RC
authors.lastname: Smith
mesh_terms: D000465:Algorithms; D016477:Artifacts; D001185:Artificial Intelligence; D001943:Breast Neoplasms / Q000000981:diagnostic imaging*; D005260:Female; D006801:Humans; D008327:Mammography / Q000379:methods*; D015588:Observer Variation; D010363:Pattern Recognition, Automated / Q000379:methods*; D011856:Radiographic Image Enhancement / Q000379:methods; D011857:Radiographic Image Interpretation, Computer-Assisted / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D012984:Software; D012986:Software Validation