Abstract coding of audiovisual speech: beyond sensory representation.
Is there a neural representation of speech that transcends its sensory properties? Using fMRI, we investigated whether there are brain areas where neural activity during observation of sublexical audiovisual input corresponds to a listener's speech percept (what is "heard") independent of the sensory properties of the input. A target audiovisual stimulus was preceded by stimuli that (1) shared the target's auditory features (auditory overlap), (2) shared the target's visual features (visual overlap), or (3) shared neither the target's auditory or visual features but were perceived as the target (perceptual overlap). In two left-hemisphere regions (pars opercularis, planum polare), the target invoked less activity when it was preceded by the perceptually overlapping stimulus than when preceded by stimuli that shared one of its sensory components. This pattern of neural facilitation indicates that these regions code sublexical speech at an abstract level corresponding to that of the speech percept.
Hasson
Uri
U
Department of Neurology, The University of Chicago, Chicago, IL 60637, USA. uhasson@uchicago.edu
Skipper
Jeremy I
JI
Nusbaum
Howard C
HC
Small
Steven L
SL
D000161:Acoustic Stimulation / Q000379:methods; D001921:Brain / Q000098:blood supply / Q000502:physiology*; D001931:Brain Mapping; D004569:Electroencephalography; D007839:Functional Laterality / Q000502:physiology; D006801:Humans; D007091:Image Processing, Computer-Assisted / Q000379:methods; D007206:Individuality; D008279:Magnetic Resonance Imaging / Q000379:methods; D008606:Mental Processes; D010100:Oxygen / Q000097:blood; D010364:Pattern Recognition, Visual / Q000502:physiology*; D010775:Photic Stimulation / Q000379:methods; D011930:Reaction Time; D013067:Speech Perception / Q000502:physiology*
D010100:Oxygen