Binding symbols and sounds: evidence from event-related oscillatory gamma-band activity.
The present study intended to examine the neural basis of audiovisual integration, hypothetically achieved by synchronized gamma-band oscillations (30-80 Hz) that have been suggested to integrate stimulus features and top-down information. To that end, we studied the impact of visual symbolic information on early auditory sensory processing of upcoming sounds. In particular, we used a symbol-to-sound-matching paradigm in which simple score-like patterns predict corresponding sound patterns. Occasionally, a single sound is incongruent with the corresponding element of the visual pattern. In response to expected sounds congruent with the corresponding visual symbol, a power increase of phase-locked (evoked) activity in the 40-Hz band was observed peaking 42-ms poststimulus onset. Thus, for the first time, we demonstrated that the comparison process between a neural model, the expectation, and the current sensory input is implemented at very early levels of auditory processing. Subsequently, expected congruent sounds elicited a broadband power increase of non-phase-locked (induced) activity peaking 152-ms poststimulus onset, which might reflect the formation of a unitary event representation including both visual and auditory aspects of the stimulation. Gamma-band responses were not present for unexpected incongruent sounds. A model explaining the anticipatory activation of cortical auditory representations and the match of experience against expectation is presented.
Widmann
Andreas
A
Institute of Psychology I, University of Leipzig, Seeburgstrasse 14-20, D-04103, Leipzig, Germany. widmann@uni-leipzig.de
Gruber
Thomas
T
Kujala
Teija
T
Tervaniemi
Mari
M
Schröger
Erich
E
D000161:Acoustic Stimulation / Q000379:methods*; D000328:Adult; D001245:Association Learning / Q000502:physiology*; D001683:Biological Clocks / Q000502:physiology*; D001921:Brain / Q000502:physiology*; D004569:Electroencephalography / Q000379:methods; D005072:Evoked Potentials, Auditory / Q000502:physiology; D005074:Evoked Potentials, Visual / Q000502:physiology; D005260:Female; D006801:Humans; D008297:Male; D046709:Pattern Recognition, Physiological / Q000502:physiology*; D010364:Pattern Recognition, Visual / Q000502:physiology*; D010775:Photic Stimulation / Q000379:methods*