Hearing what the eyes see: auditory encoding of visual temporal sequences.
When the senses deliver conflicting information, vision dominates spatial processing, and audition dominates temporal processing. We asked whether this sensory specialization results in cross-modal encoding of unisensory input into the task-appropriate modality. Specifically, we investigated whether visually portrayed temporal structure receives automatic, obligatory encoding in the auditory domain. In three experiments, observers judged whether the changes in two successive visual sequences followed the same or different rhythms. We assessed temporal representations by measuring the extent to which both task-irrelevant auditory information and task-irrelevant visual information interfered with rhythm discrimination. Incongruent auditory information significantly disrupted task performance, particularly when presented during encoding; by contrast, varying the nature of the rhythm-depicting visual changes had minimal impact on performance. Evidently, the perceptual system automatically and obligatorily abstracts temporal structure from its visual form and represents this structure using an auditory code, resulting in the experience of "hearing visual rhythms."
Guttman
Sharon E
SE
Department of Psychology, Vanderbilt University, 111 21st Avenue South, 301 Wilson Hall, Nashville, TN 37203, USA. sharon.guttman@vanderbilt.edu
Gilroy
Lee A
LA
Blake
Randolph
R
D000161:Acoustic Stimulation; D001288:Attention; D001307:Auditory Perception; D004193:Discrimination Learning; D006801:Humans; D009146:Music; D010364:Pattern Recognition, Visual; D010775:Photic Stimulation; D011571:Psychoacoustics; D011601:Psychophysics; D012691:Serial Learning; D013269:Stochastic Processes; D013998:Time Perception; D014796:Visual Perception
Curr Opin Neurobiol. 2001 Aug;11(4):505-9
11502399
Percept Psychophys. 2001 May;63(4):719-25
11436740
Brain Res Cogn Brain Res. 2002 Jun;14(1):147-52
12063138
J Exp Psychol Hum Percept Perform. 2002 Oct;28(5):1085-99
12421057
Nat Rev Neurosci. 2003 Feb;4(2):93-102
12563280
J Neurophysiol. 2003 Feb;89(2):1078-93
12574482
Percept Psychophys. 2003 May;65(4):602-12
12812282
Int J Psychophysiol. 2003 Oct;50(1-2):117-24
14511840
Int J Psychophysiol. 2003 Oct;50(1-2):125-45
14511841
Psychol Bull. 1980 Nov;88(3):638-67
7003641
Percept Psychophys. 1986 Apr;39(4):294-300
3737359
Mem Cognit. 1989 Jul;17(4):373-83
2761398
Mem Cognit. 1991 Sep;19(5):514-22
1956312
Annu Rev Neurosci. 1995;18:19-43
7605060
Nature. 1997 Jan 23;385(6614):308
9002513
Science. 1997 Apr 25;276(5312):593-6
9110978
Proc Natl Acad Sci U S A. 1998 Feb 3;95(3):869-75
9448253
J Cogn Neurosci. 1998 Nov;10(6):752-65
9831742
Science. 1964 Sep 18;145(3638):1328-30
14173429
Percept Mot Skills. 1999 Dec;89(3 Pt 2):1139-58
10710763
Mem Cognit. 2000 Jun;28(4):529-38
10946536
Nature. 2000 Dec 14;408(6814):788
11130706
Nature. 2002 Mar 14;416(6877):172-4
11894093