title: Speaker's hand gestures modulate speech perception through phase resetting of ongoing neural oscillations.
abstract: Speakers often accompany speech with spontaneous beat gestures in natural spoken communication. These gestures are usually aligned with lexical stress and can modulate the saliency of their affiliate words. Here we addressed the consequences of beat gestures on the neural correlates of speech perception. Previous studies have highlighted the role played by theta oscillations in temporal prediction of speech. We hypothesized that the sight of beat gestures may influence ongoing low-frequency neural oscillations around the onset of the corresponding words. Electroencephalographic (EEG) recordings were acquired while participants watched a continuous, naturally recorded discourse. The phase-locking value (PLV) at word onset was calculated from the EEG from pairs of identical words that had been pronounced with and without a concurrent beat gesture in the discourse. We observed an increase in PLV in the 5-6 Hz theta range as well as a desynchronization in the 8-10 Hz alpha band around the onset of words preceded by a beat gesture. These findings suggest that beats help tune low-frequency oscillatory activity at relevant moments during natural speech perception, providing a new insight of how speech and paralinguistic information are integrated.
authors.affiliation: Multisensory Research Group, Center for Brain and Cognition, Universitat Pompeu Fabra, Barcelona, Spain.
authors.forename: Emmanuel
authors.initials: E
authors.lastname: Biau
authors.affiliation: Multisensory Research Group, Center for Brain and Cognition, Universitat Pompeu Fabra, Barcelona, Spain.
authors.forename: Mireia
authors.initials: M
authors.lastname: Torralba
authors.affiliation: Department of Basic Psychology, University of Barcelona, Barcelona, Spain; Bellvitge Research Biomedical Institute (IDIBELL), Cognition and Brain Plasticity Unit, Barcelona, Spain.
authors.forename: Lluis
authors.initials: L
authors.lastname: Fuentemilla
authors.affiliation: Department of Basic Psychology, University of Barcelona, Barcelona, Spain; Bellvitge Research Biomedical Institute (IDIBELL), Cognition and Brain Plasticity Unit, Barcelona, Spain.
authors.forename: Ruth
authors.initials: R
authors.lastname: de Diego Balaguer
authors.affiliation: Multisensory Research Group, Center for Brain and Cognition, Universitat Pompeu Fabra, Barcelona, Spain; Institució Catalana de Recerca i Estudis Avançats (ICREA), Barcelona, Spain. Electronic address: salvador.soto@icrea.cat.
authors.forename: Salvador
authors.initials: S
authors.lastname: Soto-Faraco
mesh_terms: D000328:Adult; D001307:Auditory Perception; D003344:Cortical Synchronization; D004569:Electroencephalography; D005260:Female; D005868:Gestures; D006225:Hand; D006801:Humans; D008297:Male; D009633:Nonverbal Communication / Q000502:physiology* / Q000523:psychology*; D011597:Psychomotor Performance / Q000502:physiology; D013060:Speech / Q000502:physiology*; D013067:Speech Perception / Q000502:physiology*; D013826:Theta Rhythm; D014796:Visual Perception; D055815:Young Adult
keywords: Audiovisual speech; Beats; EEG; Low frequency oscillations