Processing of changes in visual speech in the human auditory cortex.
Seeing a talker's articulatory gestures may affect the observer's auditory speech percept. Observing congruent articulatory gestures may enhance the recognition of speech sounds [J. Acoust. Soc. Am. 26 (1954) 212], whereas observing incongruent gestures may change the auditory percept phonetically as occurs in the McGurk effect [Nature 264 (1976) 746]. For example, simultaneous acoustic /ba/ and visual /ga/ are usually heard as /da/. We studied cortical processing of occasional changes in audiovisual and visual speech stimuli with magnetoencephalography. In the audiovisual experiment congruent (acoustic /iti/, visual /iti/) and incongruent (acoustic /ipi/, visual /iti/) audiovisual stimuli, which were both perceived as /iti/, were presented among congruent /ipi/ (acoustic /ipi/, visual /ipi/) stimuli. In the visual experiment only the visual components of these stimuli were presented. A visual change both in audiovisual and visual experiments activated supratemporal auditory cortices bilaterally. The auditory cortex activation to a visual change occurred later in the visual than in the audiovisual experiment, suggesting that interaction between modalities accelerates the detection of visual change in speech.
Möttönen
Riikka
R
Laboratory of Computational Engineering, Helsinki University of Technology, Espoo, Finland. riikka.mottonen@hut.fi
Krause
Christina M
CM
Tiippana
Kaisa
K
Sams
Mikko
M
D000161:Acoustic Stimulation; D000328:Adult; D001303:Auditory Cortex / Q000502:physiology*; D004193:Discrimination Learning; D005260:Female; D005868:Gestures; D006801:Humans; D008279:Magnetic Resonance Imaging; D015225:Magnetoencephalography; D008297:Male; D008875:Middle Aged; D010775:Photic Stimulation; D011571:Psychoacoustics; D021641:Recognition, Psychology; D013067:Speech Perception / Q000502:physiology*; D013997:Time Factors; D014796:Visual Perception / Q000502:physiology*