Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study.
Multisensory object-recognition processes were investigated by examining the combined influence of visual and auditory inputs upon object identification--in this case, pictures and vocalizations of animals. Behaviorally, subjects were significantly faster and more accurate at identifying targets when the picture and vocalization were matched (i.e. from the same animal), than when the target was represented in only one sensory modality. This behavioral enhancement was accompanied by a modulation of the evoked potential in the latency range and general topographic region of the visual evoked N1 component, which is associated with early feature processing in the ventral visual stream. High-density topographic mapping and dipole modeling of this multisensory effect were consistent with generators in lateral occipito-temporal cortices, suggesting that auditory inputs were modulating processing in regions of the lateral occipital cortices. Both the timing and scalp topography of this modulation suggests that there are multisensory effects during what is considered to be a relatively early stage of visual object-recognition processes, and that this modulation occurs in regions of the visual system that have traditionally been held to be unisensory processing areas. Multisensory inputs also modulated the visual 'selection-negativity', an attention dependent component of the evoked potential this is usually evoked when subjects selectively attend to a particular feature of a visual stimulus.
Molholm
Sophie
S
Cognitive Neurophysiology Laboratory, Program in Cognitive Neuroscience and Schizophrenia, Nathan S. Kline Institute for Psychiatric Research, 140 Old Orangeburg Road, Orangeburg, NY 10962, USA.
Ritter
Walter
W
Javitt
Daniel C
DC
Foxe
John J
JJ
D000328:Adult; D000818:Animals; D001288:Attention / Q000502:physiology; D001307:Auditory Perception / Q000502:physiology*; D001931:Brain Mapping; D004569:Electroencephalography; D004594:Electrophysiology; D005260:Female; D006801:Humans; D008297:Male; D011597:Psychomotor Performance / Q000502:physiology; D011930:Reaction Time / Q000502:physiology; D014793:Visual Cortex / Q000502:physiology*; D014796:Visual Perception / Q000502:physiology*; D014828:Vocalization, Animal