title: Accumulation and decay of visual capture and the ventriloquism aftereffect caused by brief audio-visual disparities.
abstract: Visual capture and the ventriloquism aftereffect resolve spatial disparities of incongruent auditory visual (AV) objects by shifting auditory spatial perception to align with vision. Here, we demonstrated the distinct temporal characteristics of visual capture and the ventriloquism aftereffect in response to brief AV disparities. In a set of experiments, subjects localized either the auditory component of AV targets (A within AV) or a second sound presented at varying delays (1-20 s) after AV exposure (A2 after AV). AV targets were trains of brief presentations (1 or 20), covering a ±30° azimuthal range, and with ±8° (R or L) disparity. We found that the magnitude of visual capture generally reached its peak within a single AV pair and did not dissipate with time, while the ventriloquism aftereffect accumulated with repetitions of AV pairs and dissipated with time. Additionally, the magnitude of the auditory shift induced by each phenomenon was uncorrelated across listeners and visual capture was unaffected by subsequent auditory targets, indicating that visual capture and the ventriloquism aftereffect are separate mechanisms with distinct effects on auditory spatial perception. Our results indicate that visual capture is a 'sample-and-hold' process that binds related objects and stores the combined percept in memory, whereas the ventriloquism aftereffect is a 'leaky integrator' process that accumulates with experience and decays with time to compensate for cross-modal disparities.
authors.affiliation: Department of Biomedical Engineering, University of Rochester, Rochester, NY, USA. adam.bosen@boystown.org.
authors.forename: Adam K
authors.initials: AK
authors.lastname: Bosen
authors.affiliation: Department of Neuroscience, University of Rochester, Rochester, NY, USA.
authors.forename: Justin T
authors.initials: JT
authors.lastname: Fleming
authors.affiliation: Department of Neuroscience, University of Rochester, Rochester, NY, USA.
authors.forename: Paul D
authors.initials: PD
authors.lastname: Allen
authors.affiliation: Department of Neuroscience, University of Rochester, Rochester, NY, USA.
authors.forename: William E
authors.initials: WE
authors.lastname: O'Neill
authors.affiliation: Department of Neuroscience, University of Rochester, Rochester, NY, USA.
authors.forename: Gary D
authors.initials: GD
authors.lastname: Paige
mesh_terms: D000161:Acoustic Stimulation; D000293:Adolescent; D000328:Adult; D000704:Analysis of Variance; D005260:Female; D006801:Humans; D008297:Male; D008568:Memory / Q000502:physiology; D010775:Photic Stimulation; D013017:Sound Localization / Q000502:physiology*; D015357:Vision Disparity / Q000502:physiology*; D014796:Visual Perception / Q000502:physiology*; D055815:Young Adult
keywords: Auditory; Cross-modal; Ventriloquism; Ventriloquism aftereffect; Visual; Visual capture