title: Inside Speech: Multisensory and Modality-specific Processing of Tongue and Lip Speech Actions.
abstract: Action recognition has been found to rely not only on sensory brain areas but also partly on the observer's motor system. However, whether distinct auditory and visual experiences of an action modulate sensorimotor activity remains largely unknown. In the present sparse sampling fMRI study, we determined to which extent sensory and motor representations interact during the perception of tongue and lip speech actions. Tongue and lip speech actions were selected because tongue movements of our interlocutor are accessible via their impact on speech acoustics but not visible because of its position inside the vocal tract, whereas lip movements are both "audible" and visible. Participants were presented with auditory, visual, and audiovisual speech actions, with the visual inputs related to either a sagittal view of the tongue movements or a facial view of the lip movements of a speaker, previously recorded by an ultrasound imaging system and a video camera. Although the neural networks involved in visual visuolingual and visuofacial perception largely overlapped, stronger motor and somatosensory activations were observed during visuolingual perception. In contrast, stronger activity was found in auditory and visual cortices during visuofacial perception. Complementing these findings, activity in the left premotor cortex and in visual brain areas was found to correlate with visual recognition scores observed for visuolingual and visuofacial speech stimuli, respectively, whereas visual activity correlated with RTs for both stimuli. These results suggest that unimodal and multimodal processing of lip and tongue speech actions rely on common sensorimotor brain areas. They also suggest that visual processing of audible but not visible movements induces motor and visual mental simulation of the perceived actions to facilitate recognition and/or to learn the association between auditory and visual signals.
authors.affiliation: CNRS UMR 5216 & Grenoble Université.
authors.forename: Avril
authors.initials: A
authors.lastname: Treille
authors.affiliation: CNRS UMR 5216 & Grenoble Université.
authors.forename: Coriandre
authors.initials: C
authors.lastname: Vilain
authors.affiliation: CNRS UMR 5216 & Grenoble Université.
authors.forename: Thomas
authors.initials: T
authors.lastname: Hueber
authors.affiliation: Université Grenoble-Alpes & CHU de Grenoble.|CNRS UMS 3552, Grenoble, France.
authors.forename: Laurent
authors.initials: L
authors.lastname: Lamalle
authors.affiliation: CNRS UMR 7309 & Aix-Marseille Université.
authors.forename: Marc
authors.initials: M
authors.lastname: Sato
mesh_terms: D000161:Acoustic Stimulation / Q000379:methods; D000293:Adolescent; D000328:Adult; D001921:Brain / Q000000981:diagnostic imaging / Q000502:physiology*; D001931:Brain Mapping; D000066499:Facial Recognition / Q000502:physiology*; D005260:Female; D006801:Humans; D008279:Magnetic Resonance Imaging; D008297:Male; D009039:Motion Perception / Q000502:physiology*; D009483:Neuropsychological Tests; D010775:Photic Stimulation / Q000379:methods; D011930:Reaction Time; D012938:Social Perception; D013067:Speech Perception / Q000502:physiology*; D055815:Young Adult