Filling-in visual motion with sounds.
Information about the motion of objects can be extracted by multiple sensory modalities, and, as a consequence, object motion perception typically involves the integration of multi-sensory information. Often, in naturalistic settings, the flow of such information can be rather discontinuous (e.g. a cat racing through the furniture in a cluttered room is partly seen and partly heard). This study addressed audio-visual interactions in the perception of time-sampled object motion by measuring adaptation after-effects. We found significant auditory after-effects following adaptation to unisensory auditory and visual motion in depth, sampled at 12.5 Hz. The visually induced (cross-modal) auditory motion after-effect was eliminated if visual adaptors flashed at half of the rate (6.25 Hz). Remarkably, the addition of the high-rate acoustic flutter (12.5 Hz) to this ineffective, sparsely time-sampled, visual adaptor restored the auditory after-effect to a level comparable to what was seen with high-rate bimodal adaptors (flashes and beeps). Our results suggest that this auditory-induced reinstatement of the motion after-effect from the poor visual signals resulted from the occurrence of sound-induced illusory flashes. This effect was found to be dependent both on the directional congruency between modalities and on the rate of auditory flutter. The auditory filling-in of time-sampled visual motion supports the feasibility of using reduced frame rate visual content in multisensory broadcasting and virtual reality applications.
Väljamäe
A
A
Laboratory for Synthetic Perceptive, Emotive and Cognitive Systems (SPECS), Institute of Audiovisual Studies, Universitat Pompeu Fabra, Tànger, 135, Barcelona 08018, Spain. aleksander.valjamae@iua.upf.edu
Soto-Faraco
S
S
D000328:Adult; D001288:Attention; D001307:Auditory Perception; D003867:Depth Perception; D005260:Female; D005366:Figural Aftereffect; D005425:Flicker Fusion; D006801:Humans; D008297:Male; D009039:Motion Perception; D009949:Orientation; D010364:Pattern Recognition, Visual; D011601:Psychophysics; D012684:Sensory Thresholds; D013017:Sound Localization