title: Predicting the Valence of a Scene from Observers' Eye Movements.
abstract: Multimedia analysis benefits from understanding the emotional content of a scene in a variety of tasks such as video genre classification and content-based image retrieval. Recently, there has been an increasing interest in applying human bio-signals, particularly eye movements, to recognize the emotional gist of a scene such as its valence. In order to determine the emotional category of images using eye movements, the existing methods often learn a classifier using several features that are extracted from eye movements. Although it has been shown that eye movement is potentially useful for recognition of scene valence, the contribution of each feature is not well-studied. To address the issue, we study the contribution of features extracted from eye movements in the classification of images into pleasant, neutral, and unpleasant categories. We assess ten features and their fusion. The features are histogram of saccade orientation, histogram of saccade slope, histogram of saccade length, histogram of saccade duration, histogram of saccade velocity, histogram of fixation duration, fixation histogram, top-ten salient coordinates, and saliency map. We utilize machine learning approach to analyze the performance of features by learning a support vector machine and exploiting various feature fusion schemes. The experiments reveal that 'saliency map', 'fixation histogram', 'histogram of fixation duration', and 'histogram of saccade slope' are the most contributing features. The selected features signify the influence of fixation information and angular behavior of eye movements in the recognition of the valence of images.
authors.affiliation: Center for Machine Vision Research, Department of Computer Science and Engineering, University of Oulu, Oulu, Finland.
authors.forename: Hamed
authors.initials: H
authors.lastname: R-Tavakoli
authors.affiliation: Autonomous Systems and Advance Robotics Research Centre, University of Salford Manchester, Salford, United Kingdom; School of Computer Science, Engineering and Mathematics, Flinders University of South Australia, Adelaide, Australia.
authors.forename: Adham
authors.initials: A
authors.lastname: Atyabi
authors.affiliation: Learning Research Laboratory (LearnLab), P.O. Box 2000, 90014 University of Oulu, Oulu, Finland.
authors.forename: Antti
authors.initials: A
authors.lastname: Rantanen
authors.affiliation: Learning Research Laboratory (LearnLab), P.O. Box 2000, 90014 University of Oulu, Oulu, Finland.
authors.forename: Seppo J
authors.initials: SJ
authors.lastname: Laukka
authors.affiliation: Autonomous Systems and Advance Robotics Research Centre, University of Salford Manchester, Salford, United Kingdom.
authors.forename: Samia
authors.initials: S
authors.lastname: Nefti-Meziani
authors.affiliation: Center for Machine Vision Research, Department of Computer Science and Engineering, University of Oulu, Oulu, Finland.
authors.forename: Janne
authors.initials: J
authors.lastname: Heikkilä
mesh_terms: D001288:Attention / Q000502:physiology; D004644:Emotions / Q000502:physiology*; D005133:Eye Movements / Q000502:physiology*; D005260:Female; D005403:Fixation, Ocular / Q000502:physiology; D006801:Humans; D008297:Male; D015588:Observer Variation; D010775:Photic Stimulation; D021641:Recognition, Psychology / Q000502:physiology; D014584:User-Computer Interface; D014796:Visual Perception / Q000502:physiology*