title: Hearing what the eyes see: auditory encoding of visual temporal sequences.
abstract: When the senses deliver conflicting information, vision dominates spatial processing, and audition dominates temporal processing. We asked whether this sensory specialization results in cross-modal encoding of unisensory input into the task-appropriate modality. Specifically, we investigated whether visually portrayed temporal structure receives automatic, obligatory encoding in the auditory domain. In three experiments, observers judged whether the changes in two successive visual sequences followed the same or different rhythms. We assessed temporal representations by measuring the extent to which both task-irrelevant auditory information and task-irrelevant visual information interfered with rhythm discrimination. Incongruent auditory information significantly disrupted task performance, particularly when presented during encoding; by contrast, varying the nature of the rhythm-depicting visual changes had minimal impact on performance. Evidently, the perceptual system automatically and obligatorily abstracts temporal structure from its visual form and represents this structure using an auditory code, resulting in the experience of "hearing visual rhythms."
authors.affiliation: Department of Psychology, Vanderbilt University, 111 21st Avenue South, 301 Wilson Hall, Nashville, TN 37203, USA. sharon.guttman@vanderbilt.edu
authors.forename: Sharon E
authors.initials: SE
authors.lastname: Guttman
authors.forename: Lee A
authors.initials: LA
authors.lastname: Gilroy
authors.forename: Randolph
authors.initials: R
authors.lastname: Blake
mesh_terms: D000161:Acoustic Stimulation; D001288:Attention; D001307:Auditory Perception; D004193:Discrimination Learning; D006801:Humans; D009146:Music; D010364:Pattern Recognition, Visual; D010775:Photic Stimulation; D011571:Psychoacoustics; D011601:Psychophysics; D012691:Serial Learning; D013269:Stochastic Processes; D013998:Time Perception; D014796:Visual Perception
references.citation: Curr Opin Neurobiol. 2001 Aug;11(4):505-9
references.pmid: 11502399
references.citation: Percept Psychophys. 2001 May;63(4):719-25
references.pmid: 11436740
references.citation: Brain Res Cogn Brain Res. 2002 Jun;14(1):147-52
references.pmid: 12063138
references.citation: J Exp Psychol Hum Percept Perform. 2002 Oct;28(5):1085-99
references.pmid: 12421057
references.citation: Nat Rev Neurosci. 2003 Feb;4(2):93-102
references.pmid: 12563280
references.citation: J Neurophysiol. 2003 Feb;89(2):1078-93
references.pmid: 12574482
references.citation: Percept Psychophys. 2003 May;65(4):602-12
references.pmid: 12812282
references.citation: Int J Psychophysiol. 2003 Oct;50(1-2):117-24
references.pmid: 14511840
references.citation: Int J Psychophysiol. 2003 Oct;50(1-2):125-45
references.pmid: 14511841
references.citation: Psychol Bull. 1980 Nov;88(3):638-67
references.pmid: 7003641
references.citation: Percept Psychophys. 1986 Apr;39(4):294-300
references.pmid: 3737359
references.citation: Mem Cognit. 1989 Jul;17(4):373-83
references.pmid: 2761398
references.citation: Mem Cognit. 1991 Sep;19(5):514-22
references.pmid: 1956312
references.citation: Annu Rev Neurosci. 1995;18:19-43
references.pmid: 7605060
references.citation: Nature. 1997 Jan 23;385(6614):308
references.pmid: 9002513
references.citation: Science. 1997 Apr 25;276(5312):593-6
references.pmid: 9110978
references.citation: Proc Natl Acad Sci U S A. 1998 Feb 3;95(3):869-75
references.pmid: 9448253
references.citation: J Cogn Neurosci. 1998 Nov;10(6):752-65
references.pmid: 9831742
references.citation: Science. 1964 Sep 18;145(3638):1328-30
references.pmid: 14173429
references.citation: Percept Mot Skills. 1999 Dec;89(3 Pt 2):1139-58
references.pmid: 10710763
references.citation: Mem Cognit. 2000 Jun;28(4):529-38
references.pmid: 10946536
references.citation: Nature. 2000 Dec 14;408(6814):788
references.pmid: 11130706
references.citation: Nature. 2002 Mar 14;416(6877):172-4
references.pmid: 11894093