title: Cortical integration of audio-visual speech and non-speech stimuli.
abstract: Using fMRI we investigated the neural basis of audio-visual processing of speech and non-speech stimuli using physically similar auditory stimuli (speech and sinusoidal tones) and visual stimuli (animated circles and ellipses). Relative to uni-modal stimuli, the different multi-modal stimuli showed increased activation in largely non-overlapping areas. Ellipse-Speech, which most resembles naturalistic audio-visual speech, showed higher activation in the right inferior frontal gyrus, fusiform gyri, left posterior superior temporal sulcus, and lateral occipital cortex. Circle-Tone, an arbitrary audio-visual pairing with no speech association, activated middle temporal gyri and lateral occipital cortex. Circle-Speech showed activation in lateral occipital cortex, and Ellipse-Tone did not show increased activation relative to uni-modal stimuli. Further analysis revealed that middle temporal regions, although identified as multi-modal only in the Circle-Tone condition, were more strongly active to Ellipse-Speech or Circle-Speech, but regions that were identified as multi-modal for Ellipse-Speech were always strongest for Ellipse-Speech. Our results suggest that combinations of auditory and visual stimuli may together be processed by different cortical networks, depending on the extent to which multi-modal speech or non-speech percepts are evoked.
authors: Yale Child Study Center, Yale University School of Medicine, Yale University, United States. brent.vanderwyk@yale.edu
authors: Brent C
authors: BC
authors: Vander Wyk
authors: Gordon J
authors: GJ
authors: Ramsay
authors: Caitlin M
authors: CM
authors: Hudac
authors: Warren
authors: W
authors: Jones
authors: David
authors: D
authors: Lin
authors: Ami
authors: A
authors: Klin
authors: Su Mei
authors: SM
authors: Lee
authors: Kevin A
authors: KA
authors: Pelphrey
mesh_terms: D000161:Acoustic Stimulation; D000328:Adult; D000704:Analysis of Variance; D001307:Auditory Perception / Q000502:physiology*; D001931:Brain Mapping; D002540:Cerebral Cortex / Q000502:physiology*; D006801:Humans; D007091:Image Processing, Computer-Assisted; D008279:Magnetic Resonance Imaging; D013060:Speech; D014796:Visual Perception / Q000502:physiology*