title: Natural stimuli from three coherent modalities enhance behavioral responses and electrophysiological cortical activity in humans.
abstract: Cues that involve a number of sensory modalities are processed in the brain in an interactive multimodal manner rather than independently for each modality. We studied multimodal integration in a natural, yet fully controlled scene, implemented as an interactive game in an auditory-haptic-visual virtual environment. In this imitation of a natural scene, the targets of perception were ecologically valid uni-, bi- and tri-modal manifestations of a simple event-a ball hitting a wall. Subjects were engaged in the game while their behavioral and early cortical electrophysiological responses were measured. Behavioral results confirmed that tri-modal cues were detected faster and more accurately than bi-modal cues, which, likewise, showed advantages over unimodal responses. Event-Related Potentials (ERPs) were recorded, and the first 200 ms following stimulus onset was analyzed to reveal the latencies of cortical multimodal interactions as estimated by sLORETA. These electrophysiological findings indicated bi-modal as well as tri-modal interactions beginning very early (~30 ms), uniquely for each multimodal combination. The results suggest that early cortical multimodal integration accelerates cortical activity and, in turn, enhances performance measures. This acceleration registers on the scalp as sub-additive cortical activation.
authors: The Virtual Reality and NeuroCognition Laboratory, Technion, Israel Institute of Science, Israel; Evoked Potentials Laboratory, Technion, Israel Institute of Science, Israel.
authors: Irit
authors: I
authors: Sella
authors: The Virtual Reality and NeuroCognition Laboratory, Technion, Israel Institute of Science, Israel. Electronic address: miriamr@technion.ac.il.
authors: Miriam
authors: M
authors: Reiner
authors: Evoked Potentials Laboratory, Technion, Israel Institute of Science, Israel.
authors: Hillel
authors: H
authors: Pratt
mesh_terms: D000161:Acoustic Stimulation; D002540:Cerebral Cortex / Q000502:physiology*; D003463:Cues; D004569:Electroencephalography / Q000523:psychology*; D005071:Evoked Potentials / Q000502:physiology; D005260:Female; D006801:Humans; D008297:Male; D010775:Photic Stimulation; D011597:Psychomotor Performance / Q000502:physiology; D011930:Reaction Time / Q000502:physiology; D014584:User-Computer Interface; D018910:Video Games; D055815:Young Adult
keywords: Audition; Event-Related Potentials; Evoked potentials; Haptics; Human sensory integration; Source estimations; Virtual reality; Vision