title: Semantic association of ecologically unrelated synchronous audio-visual information in cognitive integration: an event-related potential study.
abstract: In this paper, we aimed to study the semantic association of ecologically unrelated synchronous audio-visual information in cognitive integration. A moving particle, which speed varied, was taken as a visual stimulus, while a simple tone, which frequency varied, was used as an auditory stimulus, both were synchronously presented to subjects in the form of a video. Behavioral results confirmed our hypothesis that the moving particle with varied speed and the simple tone with varied frequency were highly associated. Event-related potential (ERP) results showed that an N400 effect and a late posterior negativity (LPN) were elicited under the Incongruent condition as compared to the Congruent condition. It was further determined that there was semantic association between ecologically unrelated synchronous audio-visual information in cognitive integration. We considered that the N400 effect in our results reflected the process that stimulus-driven activities are bound together through a temporal semantic network (TSN) to form multimodal representations, while the state of this temporal semantic network was determined by both long-term learned association among stimuli and short-term experience of incoming information. The LPN might reflect the process that the human brain searches and retrieves context-specifying information in order to make a judgment, and the context-specifying information might have originated from the long-term learned association stored in the brain.
authors: Department of Computer Science and Technology, Center of Neural and Cognitive Computation, Tsinghua University, Beijing 100084, PR China. liubaolin@tsinghua.edu.cn
authors: B
authors: B
authors: Liu
authors: G
authors: G
authors: Wu
authors: Z
authors: Z
authors: Wang
authors: X
authors: X
authors: Meng
authors: Q
authors: Q
authors: Wang
mesh_terms: D000161:Acoustic Stimulation; D001307:Auditory Perception / Q000502:physiology; D001921:Brain / Q000502:physiology*; D003071:Cognition / Q000502:physiology*; D005071:Evoked Potentials / Q000502:physiology*; D005260:Female; D006801:Humans; D008297:Male; D010775:Photic Stimulation; D012660:Semantics; D012815:Signal Processing, Computer-Assisted; D014796:Visual Perception / Q000502:physiology; D055815:Young Adult