title: Multistage audiovisual integration of speech: dissociating identification and detection.
abstract: Speech perception integrates auditory and visual information. This is evidenced by the McGurk illusion where seeing the talking face influences the auditory phonetic percept and by the audiovisual detection advantage where seeing the talking face influences the detectability of the acoustic speech signal. Here, we show that identification of phonetic content and detection can be dissociated as speech-specific and non-specific audiovisual integration effects. To this end, we employed synthetically modified stimuli, sine wave speech (SWS), which is an impoverished speech signal that only observers informed of its speech-like nature recognize as speech. While the McGurk illusion only occurred for informed observers, the audiovisual detection advantage occurred for naïve observers as well. This finding supports a multistage account of audiovisual integration of speech in which the many attributes of the audiovisual speech signal are integrated by separate integration processes.
authors.affiliation: Center for Visual Cognition, Department of Psychology, University of Copenhagen, Øster Farimagsgade 2A, 1353 Copenhagen K, Denmark. kaspereskelund@gmail.com
authors.forename: Kasper
authors.initials: K
authors.lastname: Eskelund
authors.forename: Jyrki
authors.initials: J
authors.lastname: Tuomainen
authors.forename: Tobias S
authors.initials: TS
authors.lastname: Andersen
mesh_terms: D000161:Acoustic Stimulation / Q000379:methods*; D000328:Adult; D001296:Audiovisual Aids; D001307:Auditory Perception / Q000502:physiology; D005260:Female; D006801:Humans; D008297:Male; D010700:Phonetics; D010775:Photic Stimulation / Q000379:methods*; D011597:Psychomotor Performance / Q000502:physiology*; D013067:Speech Perception / Q000502:physiology*; D014796:Visual Perception / Q000502:physiology*; D055815:Young Adult