Cortical integration of audio-visual speech and non-speech stimuli.
Using fMRI we investigated the neural basis of audio-visual processing of speech and non-speech stimuli using physically similar auditory stimuli (speech and sinusoidal tones) and visual stimuli (animated circles and ellipses). Relative to uni-modal stimuli, the different multi-modal stimuli showed increased activation in largely non-overlapping areas. Ellipse-Speech, which most resembles naturalistic audio-visual speech, showed higher activation in the right inferior frontal gyrus, fusiform gyri, left posterior superior temporal sulcus, and lateral occipital cortex. Circle-Tone, an arbitrary audio-visual pairing with no speech association, activated middle temporal gyri and lateral occipital cortex. Circle-Speech showed activation in lateral occipital cortex, and Ellipse-Tone did not show increased activation relative to uni-modal stimuli. Further analysis revealed that middle temporal regions, although identified as multi-modal only in the Circle-Tone condition, were more strongly active to Ellipse-Speech or Circle-Speech, but regions that were identified as multi-modal for Ellipse-Speech were always strongest for Ellipse-Speech. Our results suggest that combinations of auditory and visual stimuli may together be processed by different cortical networks, depending on the extent to which multi-modal speech or non-speech percepts are evoked.
Vander Wyk
Brent C
BC
Yale Child Study Center, Yale University School of Medicine, Yale University, United States. brent.vanderwyk@yale.edu
Ramsay
Gordon J
GJ
Hudac
Caitlin M
CM
Jones
Warren
W
Lin
David
D
Klin
Ami
A
Lee
Su Mei
SM
Pelphrey
Kevin A
KA
D000161:Acoustic Stimulation; D000328:Adult; D000704:Analysis of Variance; D001307:Auditory Perception / Q000502:physiology*; D001931:Brain Mapping; D002540:Cerebral Cortex / Q000502:physiology*; D006801:Humans; D007091:Image Processing, Computer-Assisted; D008279:Magnetic Resonance Imaging; D013060:Speech; D014796:Visual Perception / Q000502:physiology*