title: Eyetracking for two-person tasks with manipulation of a virtual world.
abstract: Eyetracking facilities are typically restricted to monitoring a single person viewing static images or pre-recorded video. In the present article, we describe a system that makes it possible to study visual attention in coordination with other activity during joint action. The software links two eyetracking systems in parallel and provides an on-screen task. By locating eye movements against dynamic screen regions, it permits automatic tracking of moving on-screen objects. Using existing SR technology, the system can also cross-project each participant's eyetrack and mouse location onto the other's on-screen work space. Keeping a complete record of eyetrack and on-screen events in the same format as subsequent human coding, the system permits the analysis of multiple modalities. The software offers new approaches to spontaneous multimodal communication: joint action and joint attention. These capacities are demonstrated using an experimental paradigm for cooperative on-screen assembly of a two-dimensional model. The software is available under an open source license.
authors.affiliation: Human Communication Research Centre, Informatics Forum, University of Edinburgh, 10 Crichton Street, Edinburgh, Scotland. j.carletta@ed.ac.uk
authors.forename: Jean
authors.initials: J
authors.lastname: Carletta
authors.forename: Robin L
authors.initials: RL
authors.lastname: Hill
authors.forename: Craig
authors.initials: C
authors.lastname: Nicol
authors.forename: Tim
authors.initials: T
authors.lastname: Taylor
authors.forename: Jan Peter
authors.initials: JP
authors.lastname: de Ruiter
authors.forename: Ellen Gurman
authors.initials: EG
authors.lastname: Bard
mesh_terms: D005133:Eye Movements / Q000502:physiology*; D005403:Fixation, Ocular; D006801:Humans; D014584:User-Computer Interface; D014796:Visual Perception