title: Audio-visual synchrony modulates the ventriloquist illusion and its neural/spatial representation in the auditory cortex.
abstract: An essential task of our perceptual systems is to bind together the distinctive features of single objects and events into unitary percepts, even when those features are registered in different sensory modalities. In cases where auditory and visual inputs are spatially incongruent, they may still be perceived as belonging to a single event at the location of the visual stimulus - a phenomenon known as the 'ventriloquist illusion'. The present study examined how audio-visual temporal congruence influences the ventriloquist illusion and characterized its neural underpinnings with functional magnetic resonance imaging (fMRI). Behaviorally, the ventriloquist illusion was reduced for asynchronous versus synchronous audio-visual stimuli, in accordance with previous reports. Neural activity patterns associated with the ventriloquist effect were consistently observed in the planum temporale (PT), with a reduction in illusion-related fMRI-signals ipsilateral to visual stimulation for central sounds perceived peripherally and a contralateral increase in illusion-related fMRI-signals for peripheral sounds perceived centrally. Moreover, it was found that separate but adjacent regions within the PT were preferentially activated for ventriloquist illusions produced by synchronous and asynchronous audio-visual stimulation. We conclude that the left-right balance of neural activity in the PT represents the neural code that underlies the ventriloquist illusion, with greater activity in the cerebral hemisphere contralateral to the direction of the perceived shift of sound location.
authors.affiliation: Institute of Biological Psychology, Otto-von-Guericke University, 39106 Magdeburg, Germany; Department of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, Otto-von-Guericke University, 39120 Magdeburg, Germany.
authors.forename: Björn
authors.initials: B
authors.lastname: Bonath
authors.affiliation: Institute of Biological Psychology, Otto-von-Guericke University, 39106 Magdeburg, Germany; Center for Behavioral and Brain Sciences, 39120 Magdeburg, Germany.
authors.forename: Toemme
authors.initials: T
authors.lastname: Noesselt
authors.affiliation: Department of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, Otto-von-Guericke University, 39120 Magdeburg, Germany.
authors.forename: Kerstin
authors.initials: K
authors.lastname: Krauel
authors.affiliation: Institute of Cognitive Neurology and Dementia Research, Otto-von-Guericke-University, 39120 Magdeburg, Germany.
authors.forename: Sascha
authors.initials: S
authors.lastname: Tyll
authors.affiliation: Department of Neurology, Otto-von-Guericke University, 39120 Magdeburg, Germany.
authors.forename: Claus
authors.initials: C
authors.lastname: Tempelmann
authors.affiliation: Department of Neurosciences, UC San Diego, 9500 Gilman Dr. 0634, La Jolla, CA 92093, USA.
authors.forename: Steven A
authors.initials: SA
authors.lastname: Hillyard
mesh_terms: D000161:Acoustic Stimulation; D000328:Adult; D001303:Auditory Cortex / Q000502:physiology*; D001931:Brain Mapping; D005260:Female; D006801:Humans; D007088:Illusions / Q000502:physiology*; D008279:Magnetic Resonance Imaging; D008297:Male; D010775:Photic Stimulation; D013017:Sound Localization / Q000502:physiology*; D014796:Visual Perception / Q000502:physiology*; D055815:Young Adult