title: Visual speech influences speech perception immediately but not automatically.
abstract: Two experiments examined the time course of the use of auditory and visual speech cues to spoken word recognition using an eye-tracking paradigm. Results of the first experiment showed that the use of visual speech cues from lipreading is reduced if concurrently presented pictures require a division of attentional resources. This reduction was evident even when listeners' eye gaze was on the speaker rather than the (static) pictures. Experiment 2 used a deictic hand gesture to foster attention to the speaker. At the same time, the visual processing load was reduced by keeping the visual display constant over a fixed number of successive trials. Under these conditions, the visual speech cues from lipreading were used. Moreover, the eye-tracking data indicated that visual information was used immediately and even earlier than auditory information. In combination, these data indicate that visual speech cues are not used automatically, but if they are used, they are used immediately.
authors.affiliation: Department of Cognitive Science, Faculty of Media and Knowledge Sciences, University of Malta, Msida, MSD 2080, Malta. holger.mitterer@um.edu.mt.
authors.forename: Holger
authors.initials: H
authors.lastname: Mitterer
authors.affiliation: Institute of Phonetics and Speech Processing, Ludwig Maximilian University Munich, Schellingstr. 3, 80799, Munich, Germany.
authors.forename: Eva
authors.initials: E
authors.lastname: Reinisch
mesh_terms: D000161:Acoustic Stimulation / Q000379:methods; D000328:Adult; D001288:Attention / Q000502:physiology; D001307:Auditory Perception / Q000502:physiology*; D003463:Cues; D005260:Female; D006801:Humans; D008085:Lipreading; D008297:Male; D010775:Photic Stimulation / Q000379:methods; D011897:Random Allocation; D013060:Speech / Q000502:physiology*; D013067:Speech Perception / Q000502:physiology*; D014796:Visual Perception / Q000502:physiology*
keywords: Multisensory processing; Speech perception