title: Distinct cortical locations for integration of audiovisual speech and the McGurk effect.
abstract: Audiovisual (AV) speech integration is often studied using the McGurk effect, where the combination of specific incongruent auditory and visual speech cues produces the perception of a third illusory speech percept. Recently, several studies have implicated the posterior superior temporal sulcus (pSTS) in the McGurk effect; however, the exact roles of the pSTS and other brain areas in "correcting" differing AV sensory inputs remain unclear. Using functional magnetic resonance imaging (fMRI) in ten participants, we aimed to isolate brain areas specifically involved in processing congruent AV speech and the McGurk effect. Speech stimuli were composed of sounds and/or videos of consonant-vowel tokens resulting in four stimulus classes: congruent AV speech (AVCong), incongruent AV speech resulting in the McGurk effect (AVMcGurk), acoustic-only speech (AO), and visual-only speech (VO). In group- and single-subject analyses, left pSTS exhibited significantly greater fMRI signal for congruent AV speech (i.e., AVCong trials) than for both AO and VO trials. Right superior temporal gyrus, medial prefrontal cortex, and cerebellum were also identified. For McGurk speech (i.e., AVMcGurk trials), two clusters in the left posterior superior temporal gyrus (pSTG), just posterior to Heschl's gyrus or on its border, exhibited greater fMRI signal than both AO and VO trials. We propose that while some brain areas, such as left pSTS, may be more critical for the integration of AV speech, other areas, such as left pSTG, may generate the "corrected" or merged percept arising from conflicting auditory and visual cues (i.e., as in the McGurk effect). These findings are consistent with the concept that posterior superior temporal areas represent part of a "dorsal auditory stream," which is involved in multisensory integration, sensorimotor control, and optimal state estimation (Rauschecker and Scott, 2009).
authors.affiliation: Department of Neuroscience, Georgetown University Medical Center, Washington DC, USA ; Department of Neurology, Georgetown University Medical Center, Washington DC, USA.
authors.forename: Laura C
authors.initials: LC
authors.lastname: Erickson
authors.affiliation: Department of Physiology and Biophysics, Georgetown University Medical Center, Washington DC, USA ; Departments of Pediatrics and Neurology, Division of Child Neurology, University of Utah, Salt Lake City UT, USA.
authors.forename: Brandon A
authors.initials: BA
authors.lastname: Zielinski
authors.affiliation: Department of Physiology and Biophysics, Georgetown University Medical Center, Washington DC, USA.
authors.forename: Jennifer E V
authors.initials: JE
authors.lastname: Zielinski
authors.affiliation: Department of Physiology and Biophysics, Georgetown University Medical Center, Washington DC, USA ; National Institutes of Health, Bethesda MD, USA.
authors.forename: Guoying
authors.initials: G
authors.lastname: Liu
authors.affiliation: Department of Neurology, Georgetown University Medical Center, Washington DC, USA ; MedStar National Rehabilitation Hospital, Washington DC, USA.
authors.forename: Peter E
authors.initials: PE
authors.lastname: Turkeltaub
authors.affiliation: Department of Neuroscience, Georgetown University Medical Center, Washington DC, USA ; Department of Neurology, University of California Los Angeles, Los Angeles CA, USA.
authors.forename: Amber M
authors.initials: AM
authors.lastname: Leaver
authors.affiliation: Department of Neuroscience, Georgetown University Medical Center, Washington DC, USA ; Department of Physiology and Biophysics, Georgetown University Medical Center, Washington DC, USA.
authors.forename: Josef P
authors.initials: JP
authors.lastname: Rauschecker
keywords: McGurk effect; cross-modal; dorsal stream; multisensory; sensorimotor; speech; superior temporal sulcus