title: Investigating the link between radiologists' gaze, diagnostic decision, and image content.
abstract: OBJECTIVE
  To investigate machine learning for linking image content, human perception, cognition, and error in the diagnostic interpretation of mammograms.
  
  
  METHODS
  Gaze data and diagnostic decisions were collected from three breast imaging radiologists and three radiology residents who reviewed 20 screening mammograms while wearing a head-mounted eye-tracker. Image analysis was performed in mammographic regions that attracted radiologists' attention and in all abnormal regions. Machine learning algorithms were investigated to develop predictive models that link: (i) image content with gaze, (ii) image content and gaze with cognition, and (iii) image content, gaze, and cognition with diagnostic error. Both group-based and individualized models were explored.
  
  
  RESULTS
  By pooling the data from all readers, machine learning produced highly accurate predictive models linking image content, gaze, and cognition. Potential linking of those with diagnostic error was also supported to some extent. Merging readers' gaze metrics and cognitive opinions with computer-extracted image features identified 59% of the readers' diagnostic errors while confirming 97.3% of their correct diagnoses. The readers' individual perceptual and cognitive behaviors could be adequately predicted by modeling the behavior of others. However, personalized tuning was in many cases beneficial for capturing more accurately individual behavior.
  
  
  CONCLUSIONS
  There is clearly an interaction between radiologists' gaze, diagnostic decision, and image content which can be modeled with machine learning algorithms.
authors.affiliation: Oak Ridge National Laboratory, Biomedical Science and Engineering Center, Oak Ridge, Tennessee, USA.
authors.forename: Georgia
authors.initials: G
authors.lastname: Tourassi
authors.forename: Sophie
authors.initials: S
authors.lastname: Voisin
authors.forename: Vincent
authors.initials: V
authors.lastname: Paquit
authors.forename: Elizabeth
authors.initials: E
authors.lastname: Krupinski
mesh_terms: D001185:Artificial Intelligence; D001943:Breast Neoplasms / Q000000981:diagnostic imaging; D003071:Cognition; D003951:Diagnostic Errors; D005133:Eye Movements; D006801:Humans; D007091:Image Processing, Computer-Assisted; D008327:Mammography; D010865:Pilot Projects; D011871:Radiology; D014796:Visual Perception
keywords: eye-tracking; machine learning; mammography; user modeling