title: Complementary effects of gaze direction and early saliency in guiding fixations during free viewing.
abstract: Gaze direction provides an important and ubiquitous communication channel in daily behavior and social interaction of humans and some animals. While several studies have addressed gaze direction in synthesized simple scenes, few have examined how it can bias observer attention and how it might interact with early saliency during free viewing of natural and realistic scenes. Experiment 1 used a controlled, staged setting in which an actor was asked to look at two different objects in turn, yielding two images that differed only by the actor's gaze direction, to causally assess the effects of actor gaze direction. Over all scenes, the median probability of following an actor's gaze direction was higher than the median probability of looking toward the single most salient location, and higher than chance. Experiment 2 confirmed these findings over a larger set of unconstrained scenes collected from the Web and containing people looking at objects and/or other people. To further compare the strength of saliency versus gaze direction cues, we computed gaze maps by drawing a cone in the direction of gaze of the actors present in the images. Gaze maps predicted observers' fixation locations significantly above chance, although below saliency. Finally, to gauge the relative importance of actor face and eye directions in guiding observer's fixations, in Experiment 3, observers were asked to guess the gaze direction from only an actor's face region (with the rest of the scene masked), in two conditions: actor eyes visible or masked. Median probability of guessing the true gaze direction within ±9° was significantly higher when eyes were visible, suggesting that the eyes contribute significantly to gaze estimation, in addition to face region. Our results highlight that gaze direction is a strong attentional cue in guiding eye movements, complementing low-level saliency cues, and derived from both face and eyes of actors in the scene. Thus gaze direction should be considered in constructing more predictive visual attention models in the future.
authors.affiliation: Computer Science Department, University of Wisconsin, Milwaukee, WI, USA.
authors.forename: Ali
authors.initials: A
authors.lastname: Borji
authors.affiliation: Neuroscience Graduate Program, University of Southern California, Los Angeles, CA, USA.
authors.forename: Daniel
authors.initials: D
authors.lastname: Parks
authors.affiliation: Neuroscience Graduate Program, University of Southern California, Los Angeles, CA, USA Department of Computer Science, University of Southern California, Los Angeles, CA, USA Department of Psychology, University of Southern California, Los Angeles, CA, USA.
authors.forename: Laurent
authors.initials: L
authors.lastname: Itti
mesh_terms: D001288:Attention / Q000502:physiology*; D003463:Cues; D005133:Eye Movements / Q000502:physiology*; D005145:Face; D005260:Female; D005403:Fixation, Ocular / Q000502:physiology*; D006801:Humans; D008297:Male; D010364:Pattern Recognition, Visual / Q000502:physiology*; D055815:Young Adult
keywords: bottom-up saliency; eye movements; fixation prediction; free viewing; gaze cueing; gaze direction; gaze following; human–computer interaction; line of sight; overt attention; saliency modeling; top-down attention; visual attention