title: Eye'm talking to you: speakers' gaze direction modulates co-speech gesture processing in the right MTG.
abstract: Recipients process information from speech and co-speech gestures, but it is currently unknown how this processing is influenced by the presence of other important social cues, especially gaze direction, a marker of communicative intent. Such cues may modulate neural activity in regions associated either with the processing of ostensive cues, such as eye gaze, or with the processing of semantic information, provided by speech and gesture. Participants were scanned (fMRI) while taking part in triadic communication involving two recipients and a speaker. The speaker uttered sentences that were and were not accompanied by complementary iconic gestures. Crucially, the speaker alternated her gaze direction, thus creating two recipient roles: addressed (direct gaze) vs unaddressed (averted gaze) recipient. The comprehension of Speech&Gesture relative to SpeechOnly utterances recruited middle occipital, middle temporal and inferior frontal gyri, bilaterally. The calcarine sulcus and posterior cingulate cortex were sensitive to differences between direct and averted gaze. Most importantly, Speech&Gesture utterances, but not SpeechOnly utterances, produced additional activity in the right middle temporal gyrus when participants were addressed. Marking communicative intent with gaze direction modulates the processing of speech-gesture utterances in cerebral areas typically associated with the semantic processing of multi-modal communicative acts.
authors.affiliation: Max Planck Institute for Psycholinguistics, Wundtlaan 1, 6525XD Nijmegen, The Netherlands, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Geert Grooteplein Noord 21, 6525EZ Nijmegen, The Netherlands, Psychology Department, Center for Language and Brain, Colgate University, 13 Oak Drive, 13346 Hamilton, NY, USA, and Centre for Language Studies, Radboud University, Erasmusplein 1, 6525 HT Nijmegen, The Netherlands judith.holler@mpi.nl.
authors.forename: Judith
authors.initials: J
authors.lastname: Holler
authors.affiliation: Max Planck Institute for Psycholinguistics, Wundtlaan 1, 6525XD Nijmegen, The Netherlands, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Geert Grooteplein Noord 21, 6525EZ Nijmegen, The Netherlands, Psychology Department, Center for Language and Brain, Colgate University, 13 Oak Drive, 13346 Hamilton, NY, USA, and Centre for Language Studies, Radboud University, Erasmusplein 1, 6525 HT Nijmegen, The Netherlands judith.holler@mpi.nl.
authors.forename: Idil
authors.initials: I
authors.lastname: Kokal
authors.affiliation: Max Planck Institute for Psycholinguistics, Wundtlaan 1, 6525XD Nijmegen, The Netherlands, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Geert Grooteplein Noord 21, 6525EZ Nijmegen, The Netherlands, Psychology Department, Center for Language and Brain, Colgate University, 13 Oak Drive, 13346 Hamilton, NY, USA, and Centre for Language Studies, Radboud University, Erasmusplein 1, 6525 HT Nijmegen, The Netherlands.
authors.forename: Ivan
authors.initials: I
authors.lastname: Toni
authors.affiliation: Max Planck Institute for Psycholinguistics, Wundtlaan 1, 6525XD Nijmegen, The Netherlands, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Geert Grooteplein Noord 21, 6525EZ Nijmegen, The Netherlands, Psychology Department, Center for Language and Brain, Colgate University, 13 Oak Drive, 13346 Hamilton, NY, USA, and Centre for Language Studies, Radboud University, Erasmusplein 1, 6525 HT Nijmegen, The Netherlands Max Planck Institute for Psycholinguistics, Wundtlaan 1, 6525XD Nijmegen, The Netherlands, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Geert Grooteplein Noord 21, 6525EZ Nijmegen, The Netherlands, Psychology Department, Center for Language and Brain, Colgate University, 13 Oak Drive, 13346 Hamilton, NY, USA, and Centre for Language Studies, Radboud University, Erasmusplein 1, 6525 HT Nijmegen, The Netherlands.
authors.forename: Peter
authors.initials: P
authors.lastname: Hagoort
authors.affiliation: Max Planck Institute for Psycholinguistics, Wundtlaan 1, 6525XD Nijmegen, The Netherlands, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Geert Grooteplein Noord 21, 6525EZ Nijmegen, The Netherlands, Psychology Department, Center for Language and Brain, Colgate University, 13 Oak Drive, 13346 Hamilton, NY, USA, and Centre for Language Studies, Radboud University, Erasmusplein 1, 6525 HT Nijmegen, The Netherlands.
authors.forename: Spencer D
authors.initials: SD
authors.lastname: Kelly
authors.affiliation: Max Planck Institute for Psycholinguistics, Wundtlaan 1, 6525XD Nijmegen, The Netherlands, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Geert Grooteplein Noord 21, 6525EZ Nijmegen, The Netherlands, Psychology Department, Center for Language and Brain, Colgate University, 13 Oak Drive, 13346 Hamilton, NY, USA, and Centre for Language Studies, Radboud University, Erasmusplein 1, 6525 HT Nijmegen, The Netherlands Max Planck Institute for Psycholinguistics, Wundtlaan 1, 6525XD Nijmegen, The Netherlands, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Geert Grooteplein Noord 21, 6525EZ Nijmegen, The Netherlands, Psychology Department, Center for Language and Brain, Colgate University, 13 Oak Drive, 13346 Hamilton, NY, USA, and Centre for Language Studies, Radboud University, Erasmusplein 1, 6525 HT Nijmegen, The Netherlands.
authors.forename: Aslı
authors.initials: A
authors.lastname: Özyürek
mesh_terms: D001931:Brain Mapping; D003142:Communication; D032882:Comprehension; D003463:Cues; D005260:Female; D005403:Fixation, Ocular / Q000502:physiology*; D005868:Gestures; D006179:Gyrus Cinguli / Q000502:physiology*; D006801:Humans; D008279:Magnetic Resonance Imaging; D013060:Speech; D055815:Young Adult
keywords: co-speech gestures; communicative intent; eye gaze; middle temporal gyrus; speech–gesture integration