title: Involvement of superior temporal areas in audiovisual and audiomotor speech integration.
abstract: Perception of speech sounds is affected by observing facial motion. Incongruence between speech sounds and watching somebody articulating may influence the perception of auditory syllable, referred to as the McGurk effect. We tested the degree to which silent articulation of a syllable also affects speech perception and searched for its neural correlates. Listeners were instructed to identify the auditory syllables /pa/ and /ta/ while silently articulating congruent/incongruent syllables or observing videos of a speaker's face articulating them. As a baseline, we included an auditory-only condition without competing visual or sensorimotor input. As expected, perception of sounds degraded when incongruent syllables were observed, and also when they were silently articulated, albeit to a lesser extent. This degrading was accompanied by significant amplitude modulations in the beta frequency band in right superior temporal areas. In these areas, the event-related beta activity during congruent conditions was phase-locked to responses evoked during the auditory-only condition. We conclude that proper temporal alignment of different input streams in right superior temporal areas is mandatory for both audiovisual and audiomotor speech integration.
authors.affiliation: MOVE Research Institute Amsterdam, Faculty of Behavioural and Movement Sciences, Vrije Universiteit, Van der Boechorststraat 9, 1081BT Amsterdam, The Netherlands; Department of Neurological, Biomedical and Movement Sciences, University of Verona, 37131 Verona, Italy.
authors.forename: N
authors.initials: N
authors.lastname: Komeilipoor
authors.affiliation: Department of Neurological, Biomedical and Movement Sciences, University of Verona, 37131 Verona, Italy.
authors.forename: P
authors.initials: P
authors.lastname: Cesari
authors.affiliation: MOVE Research Institute Amsterdam, Faculty of Behavioural and Movement Sciences, Vrije Universiteit, Van der Boechorststraat 9, 1081BT Amsterdam, The Netherlands. Electronic address: a.daffertshofer@vu.nl.
authors.forename: A
authors.initials: A
authors.lastname: Daffertshofer
mesh_terms: D000328:Adult; D001611:Beta Rhythm; D004576:Electromyography; D000066499:Facial Recognition / Q000502:physiology*; D005260:Female; D006801:Humans; D007092:Imagination / Q000502:physiology; D008297:Male; D009483:Neuropsychological Tests; D013060:Speech / Q000502:physiology*; D013067:Speech Perception / Q000502:physiology / Q000528:radiation effects*; D013702:Temporal Lobe / Q000502:physiology*
keywords: EEG; McGurk effect; multisensory integration; sensorimotor interaction; superior temporal gyrus