title: Integration of facial and newly learned visual cues in speech perception.
abstract: We are developing technology to translate acoustic characteristics of speech into visual cues that can be used to supplement speechreading when hearing is limited. Research and theory have established that perceivers are influenced by multiple sources of sensory and contextual information in spoken language processing. Previous research has also shown that additional sources of information can be learned and used to supplement those that are normally available but have been degraded by sensory impairment or difficult environments. We tested whether people can combine or integrate information from the face and information from newly learned cues in an optimal manner. Subjects first learned the visual cues and then were tested under three conditions.Words were presented with just the face, just the visual cues, or both together. Performance was much better with both cues than with either one alone. Similar to the description of previous results with audible and visible speech, the present results were well described by the Fuzzy Logical Model of Perception (Massaro, 1998), which predicts optimal or maximally efficient integration.
authors: University of California, Santa Cruz, Department of Psychology, CA 95064, USA. massaro@usc.edu
authors: Dom
authors: D
authors: Massaro
authors: Michael M
authors: MM
authors: Cohen
authors: Heidi
authors: H
authors: Meyer
authors: Tracy
authors: T
authors: Stribling
authors: Cass
authors: C
authors: Sterling
authors: Sam
authors: S
authors: Vanderhyden
mesh_terms: D001245:Association Learning; D003118:Color Perception; D032882:Comprehension; D003463:Cues; D005139:Eyeglasses; D005145:Face; D005260:Female; D017143:Fuzzy Logic; D006310:Hearing Aids; D006801:Humans; D008085:Lipreading; D008297:Male; D010364:Pattern Recognition, Visual; D010470:Perceptual Masking; D010699:Phonation; D010700:Phonetics; D013061:Speech Acoustics; D013067:Speech Perception; D018511:Systems Integration; D055815:Young Adult