Sounds exaggerate visual shape.
While perceiving speech, people see mouth shapes that are systematically associated with sounds. In particular, a vertically stretched mouth produces a /woo/ sound, whereas a horizontally stretched mouth produces a /wee/ sound. We demonstrate that hearing these speech sounds alters how we see aspect ratio, a basic visual feature that contributes to perception of 3D space, objects and faces. Hearing a /woo/ sound increases the apparent vertical elongation of a shape, whereas hearing a /wee/ sound increases the apparent horizontal elongation. We further demonstrate that these sounds influence aspect ratio coding. Viewing and adapting to a tall (or flat) shape makes a subsequently presented symmetric shape appear flat (or tall). These aspect ratio aftereffects are enhanced when associated speech sounds are presented during the adaptation period, suggesting that the sounds influence visual population coding of aspect ratio. Taken together, these results extend previous demonstrations that visual information constrains auditory perception by showing the converse - speech sounds influence visual perception of a basic geometric feature.
Department of Psychology, University of California - Berkeley, Berkeley, CA 94720, USA. timsweeny@berkeley.edu
Timothy D
TD
Sweeny
Emmanuel
E
Guzman-Martinez
Laura
L
Ortega
Marcia
M
Grabowecky
Satoru
S
Suzuki
D000704:Analysis of Variance; D001307:Auditory Perception; D006801:Humans; D010700:Phonetics; D013067:Speech Perception; D014796:Visual Perception