title: Exploring the role of gaze behavior and object detection in scene understanding.
abstract: We posit that a person's gaze behavior while freely viewing a scene contains an abundance of information, not only about their intent and what they consider to be important in the scene, but also about the scene's content. Experiments are reported, using two popular image datasets from computer vision, that explore the relationship between the fixations that people make during scene viewing, how they describe the scene, and automatic detection predictions of object categories in the scene. From these exploratory analyses, we then combine human behavior with the outputs of current visual recognition methods to build prototype human-in-the-loop applications for gaze-enabled object detection and scene annotation.
authors.affiliation: Computer Science Department, Stony Brook University Stony Brook, NY, USA.
authors.forename: Kiwon
authors.initials: K
authors.lastname: Yun
authors.affiliation: Computer Science Department, Stony Brook University Stony Brook, NY, USA.
authors.forename: Yifan
authors.initials: Y
authors.lastname: Peng
authors.affiliation: Computer Science Department, Stony Brook University Stony Brook, NY, USA.
authors.forename: Dimitris
authors.initials: D
authors.lastname: Samaras
authors.affiliation: Computer Science Department, Stony Brook University Stony Brook, NY, USA ; Psychology Department, Stony Brook University Stony Brook, NY, USA.
authors.forename: Gregory J
authors.initials: GJ
authors.lastname: Zelinsky
authors.affiliation: Department of Computer Science, University of North Carolina, Chapel-Hill Chapel Hill, NC, USA.
authors.forename: Tamara L
authors.initials: TL
authors.lastname: Berg
keywords: computer vision; description; gaze; image annotation; object detection; scene understanding