Crossmodal adaptation in right posterior superior temporal sulcus during face-voice emotional integration.
The integration of emotional information from the face and voice of other persons is known to be mediated by a number of "multisensory" cerebral regions, such as the right posterior superior temporal sulcus (pSTS). However, whether multimodal integration in these regions is attributable to interleaved populations of unisensory neurons responding to face or voice or rather by multimodal neurons receiving input from the two modalities is not fully clear. Here, we examine this question using functional magnetic resonance adaptation and dynamic audiovisual stimuli in which emotional information was manipulated parametrically and independently in the face and voice via morphing between angry and happy expressions. Healthy human adult subjects were scanned while performing a happy/angry emotion categorization task on a series of such stimuli included in a fast event-related, continuous carryover design. Subjects integrated both face and voice information when categorizing emotion-although there was a greater weighting of face information-and showed behavioral adaptation effects both within and across modality. Adaptation also occurred at the neural level: in addition to modality-specific adaptation in visual and auditory cortices, we observed for the first time a crossmodal adaptation effect. Specifically, fMRI signal in the right pSTS was reduced in response to a stimulus in which facial emotion was similar to the vocal emotion of the preceding stimulus. These results suggest that the integration of emotional information from face and voice in the pSTS involves a detectable proportion of bimodal neurons that combine inputs from visual and auditory cortices.
Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht 6229 EV, The Netherlands, Centre for Cognitive Neuroimaging, Institute of Neuroscience and Psychology, University of Glasgow, Glasgow, G12 8QB, United Kingdom, rebecca.watson@maastrichtuniversity.nl.
Rebecca
R
Watson
Centre for Cognitive Neuroimaging, Institute of Neuroscience and Psychology, University of Glasgow, Glasgow, G12 8QB, United Kingdom, Neuroscience Institute of Timone, Coeducational Research Unit 7289, National Center of Scientific Research-Aix-Marseille University, F-13284 Marseille, France.
Marianne
M
Latinus
Department of Psychology, University of Warwick, Coventry, CV4 7A, United Kingdom, and.
Takao
T
Noguchi
Centre for Cognitive Neuroimaging, Institute of Neuroscience and Psychology, University of Glasgow, Glasgow, G12 8QB, United Kingdom.
Oliver
O
Garrod
Centre for Cognitive Neuroimaging, Institute of Neuroscience and Psychology, University of Glasgow, Glasgow, G12 8QB, United Kingdom.
Frances
F
Crabbe
Centre for Cognitive Neuroimaging, Institute of Neuroscience and Psychology, University of Glasgow, Glasgow, G12 8QB, United Kingdom, Neuroscience Institute of Timone, Coeducational Research Unit 7289, National Center of Scientific Research-Aix-Marseille University, F-13284 Marseille, France, International Laboratories for Brain, Music, and Sound, University of Montreal and McGill University, Montreal, Quebec, Canada, H2V 4P3.
Pascal
P
Belin
D000161:Acoustic Stimulation; D000328:Adult; D001307:Auditory Perception / Q000502:physiology*; D001931:Brain Mapping; D004644:Emotions / Q000502:physiology*; D005260:Female; D006801:Humans; D007091:Image Processing, Computer-Assisted; D008279:Magnetic Resonance Imaging; D008297:Male; D010775:Photic Stimulation; D012938:Social Perception; D013702:Temporal Lobe / Q000502:physiology*; D014796:Visual Perception / Q000502:physiology*; D014831:Voice
emotion perception; functional magnetic resonance adaptation; multisensory integration