title: Speech Perception as a Multimodal Phenomenon.
abstract: Speech perception is inherently multimodal. Visual speech (lip-reading) information is used by all perceivers and readily integrates with auditory speech. Imaging research suggests that the brain treats auditory and visual speech similarly. These findings have led some researchers to consider that speech perception works by extracting amodal information that takes the same form across modalities. From this perspective, speech integration is a property of the input information itself. Amodal speech information could explain the reported automaticity, immediacy, and completeness of audiovisual speech integration. However, recent findings suggest that speech integration can be influenced by higher cognitive properties such as lexical status and semantic context. Proponents of amodal accounts will need to explain these results.
authors.affiliation: University of California, Riverside.
authors.forename: Lawrence D
authors.initials: LD
authors.lastname: Rosenblum
keywords: audiovisual; lip reading; multimodal; speech
references.citation: J Exp Psychol Hum Percept Perform. 1991 Aug;17(3):816-28
references.pmid: 1834793
references.citation: Exp Brain Res. 2006 Jan;168(1-2):1-10
references.pmid: 16217645
references.citation: Nature. 1976 Dec 23-30;264(5588):746-8
references.pmid: 1012311
references.citation: Percept Psychophys. 2005 May;67(4):580-94
references.pmid: 16134453
references.citation: J Exp Psychol Hum Percept Perform. 2004 Jun;30(3):445-63
references.pmid: 15161378
references.citation: Psychol Sci. 2007 May;18(5):392-6
references.pmid: 17576277
references.citation: Brain Res Cogn Brain Res. 2005 May;23(2-3):429-35
references.pmid: 15820649
references.citation: J Acoust Soc Am. 2000 Sep;108(3 Pt 1):1197-208
references.pmid: 11008820
references.citation: Curr Opin Neurobiol. 2001 Aug;11(4):505-9
references.pmid: 11502399
references.citation: Exp Brain Res. 2005 Nov;167(1):66-75
references.pmid: 16034571
references.citation: Science. 1997 Apr 25;276(5312):593-6
references.pmid: 9110978
references.citation: J Exp Psychol Hum Percept Perform. 1997 Jun;23(3):651-66
references.pmid: 9180039
references.citation: Cognition. 2005 May;96(1):B13-22
references.pmid: 15833302
references.citation: J Neurosci. 2005 May 18;25(20):5004-12
references.pmid: 15901781