title: Social eye gaze modulates processing of speech and co-speech gesture.
abstract: In human face-to-face communication, language comprehension is a multi-modal, situated activity. However, little is known about how we combine information from different modalities during comprehension, and how perceived communicative intentions, often signaled through visual signals, influence this process. We explored this question by simulating a multi-party communication context in which a speaker alternated her gaze between two recipients. Participants viewed speech-only or speech+gesture object-related messages when being addressed (direct gaze) or unaddressed (gaze averted to other participant). They were then asked to choose which of two object images matched the speaker's preceding message. Unaddressed recipients responded significantly more slowly than addressees for speech-only utterances. However, perceiving the same speech accompanied by gestures sped unaddressed recipients up to a level identical to that of addressees. That is, when unaddressed recipients' speech processing suffers, gestures can enhance the comprehension of a speaker's message. We discuss our findings with respect to two hypotheses attempting to account for how social eye gaze may modulate multi-modal language comprehension.
authors.affiliation: Max Planck Institute for Psycholinguistics, Wundtlaan 1, 6525 XD Nijmegen, The Netherlands; University of Manchester, School of Psychological Sciences, Coupland Building 1, M13 9PL Manchester, UK. Electronic address: judith.holler@mpi.nl.
authors.forename: Judith
authors.initials: J
authors.lastname: Holler
authors.affiliation: Max Planck Institute for Psycholinguistics, Wundtlaan 1, 6525 XD Nijmegen, The Netherlands; Max Planck Institute for Demographic Research, Konrad-Zuse-Straße 1, 18057 Rostock, Germany.
authors.forename: Louise
authors.initials: L
authors.lastname: Schubotz
authors.affiliation: Colgate University, Psychology Department, Center for Language and Brain, Oak Drive 13, Hamilton, NY 13346, USA.
authors.forename: Spencer
authors.initials: S
authors.lastname: Kelly
authors.affiliation: Max Planck Institute for Psycholinguistics, Wundtlaan 1, 6525 XD Nijmegen, The Netherlands; Radboud University, Donders Institute for Brain, Cognition and Behaviour, Montessorilaan 3, 6525 HR Nijmegen, The Netherlands.
authors.forename: Peter
authors.initials: P
authors.lastname: Hagoort
authors.affiliation: Max Planck Institute for Psycholinguistics, Wundtlaan 1, 6525 XD Nijmegen, The Netherlands.
authors.forename: Manuela
authors.initials: M
authors.lastname: Schuetze
authors.affiliation: Radboud University, Centre for Language Studies, Erasmusplein 1, 6525HT Nijmegen, The Netherlands; Max Planck Institute for Psycholinguistics, Wundtlaan 1, 6525 XD Nijmegen, The Netherlands.
authors.forename: Aslı
authors.initials: A
authors.lastname: Özyürek
mesh_terms: D003142:Communication; D032882:Comprehension; D005133:Eye Movements / Q000502:physiology*; D005260:Female; D005868:Gestures; D006801:Humans; D008297:Male; D011930:Reaction Time / Q000502:physiology; D013060:Speech / Q000502:physiology*; D013067:Speech Perception / Q000502:physiology*; D055815:Young Adult
keywords: Co-speech iconic gesture; Communicative intent; Eye gaze; Language processing; Multi-party communication; Recipient status