title: Vision of tongue movements bias auditory speech perception.
abstract: Audiovisual speech perception is likely based on the association between auditory and visual information into stable audiovisual maps. Conflicting audiovisual inputs generate perceptual illusions such as the McGurk effect. Audiovisual mismatch effects could be either driven by the detection of violations in the standard audiovisual statistics or via the sensorimotor reconstruction of the distal articulatory event that generated the audiovisual ambiguity. In order to disambiguate between the two hypotheses we exploit the fact that the tongue is hidden to vision. For this reason, tongue movement encoding can solely be learned via speech production but not via others׳ speech perception alone. Here we asked participants to identify speech sounds while matching or mismatching visual representations of tongue movements which were shown. Vision of congruent tongue movements facilitated auditory speech identification with respect to incongruent trials. This result suggests that direct visual experience of an articulator movement is not necessary for the generation of audiovisual mismatch effects. Furthermore, we suggest that audiovisual integration in speech may benefit from speech production learning.
authors.affiliation: Robotics, Brain and Cognitive Sciences Department, The Italian Institute of Technology, Via Morego, 30, 16163 Genova, Italy. Electronic address: alessandro.dausilio@iit.it.
authors.forename: Alessandro
authors.initials: A
authors.lastname: D'Ausilio
authors.affiliation: Robotics, Brain and Cognitive Sciences Department, The Italian Institute of Technology, Via Morego, 30, 16163 Genova, Italy.
authors.forename: Eleonora
authors.initials: E
authors.lastname: Bartoli
authors.affiliation: Robotics, Brain and Cognitive Sciences Department, The Italian Institute of Technology, Via Morego, 30, 16163 Genova, Italy.
authors.forename: Laura
authors.initials: L
authors.lastname: Maffongelli
authors.affiliation: Robotics, Brain and Cognitive Sciences Department, The Italian Institute of Technology, Via Morego, 30, 16163 Genova, Italy.
authors.forename: Jeffrey James
authors.initials: JJ
authors.lastname: Berry
authors.affiliation: Robotics, Brain and Cognitive Sciences Department, The Italian Institute of Technology, Via Morego, 30, 16163 Genova, Italy; Section of Human Physiology, University of Ferrara, Via Fossato di Mortara, 17/19, 44100 Ferrara, Italy.
authors.forename: Luciano
authors.initials: L
authors.lastname: Fadiga
mesh_terms: D000161:Acoustic Stimulation; D000328:Adult; D005260:Female; D006801:Humans; D008297:Male; D009068:Movement; D013060:Speech; D013067:Speech Perception; D014059:Tongue; D014796:Visual Perception; D055815:Young Adult
keywords: Action perception; Audiovisual speech perception; McGurk; Motor theories; Sensorimotor map; Speech