Eyetracking for two-person tasks with manipulation of a virtual world.
Eyetracking facilities are typically restricted to monitoring a single person viewing static images or pre-recorded video. In the present article, we describe a system that makes it possible to study visual attention in coordination with other activity during joint action. The software links two eyetracking systems in parallel and provides an on-screen task. By locating eye movements against dynamic screen regions, it permits automatic tracking of moving on-screen objects. Using existing SR technology, the system can also cross-project each participant's eyetrack and mouse location onto the other's on-screen work space. Keeping a complete record of eyetrack and on-screen events in the same format as subsequent human coding, the system permits the analysis of multiple modalities. The software offers new approaches to spontaneous multimodal communication: joint action and joint attention. These capacities are demonstrated using an experimental paradigm for cooperative on-screen assembly of a two-dimensional model. The software is available under an open source license.
Carletta
Jean
J
Human Communication Research Centre, Informatics Forum, University of Edinburgh, 10 Crichton Street, Edinburgh, Scotland. j.carletta@ed.ac.uk
Hill
Robin L
RL
Nicol
Craig
C
Taylor
Tim
T
de Ruiter
Jan Peter
JP
Bard
Ellen Gurman
EG
D005133:Eye Movements / Q000502:physiology*; D005403:Fixation, Ocular; D006801:Humans; D014584:User-Computer Interface; D014796:Visual Perception