title: [Virtual audiovisual talking heads: articulatory data and models--applications].
abstract: In the framework of experimental phonetics, our approach to the study of speech production is based on the measurement, the analysis and the modeling of orofacial articulators such as the jaw, the face and the lips, the tongue or the velum. Therefore, we present in this article experimental techniques that allow characterising the shape and movement of speech articulators (static and dynamic MRI, computed tomodensitometry, electromagnetic articulography, video recording). We then describe the linear models of the various organs that we can elaborate from speaker-specific articulatory data. We show that these models, that exhibit a good geometrical resolution, can be controlled from articulatory data with a good temporal resolution and can thus permit the reconstruction of high quality animation of the articulators. These models, that we have integrated in a virtual talking head, can produce augmented audiovisual speech. In this framework, we have assessed the natural tongue reading capabilities of human subjects by means of audiovisual perception tests. We conclude by suggesting a number of other applications of talking heads.
authors.affiliation: GIPSA-lab, ENSIEG, Domaine Universitaire, BP46, 38402 Saint Martin d'Hères, France. Pierre.Badin@gipsa-lab.inpg.fr
authors.forename: P
authors.initials: P
authors.lastname: Badin
authors.forename: F
authors.initials: F
authors.lastname: Elisei
authors.forename: G
authors.initials: G
authors.lastname: Bailly
authors.forename: C
authors.initials: C
authors.lastname: Savariaux
authors.forename: A
authors.initials: A
authors.lastname: Serrurier
authors.forename: Y
authors.initials: Y
authors.lastname: Tarabalka
mesh_terms: D005145:Face / Q000502:physiology*; D006801:Humans; D007568:Jaw / Q000502:physiology*; D016014:Linear Models; D008046:Lip / Q000502:physiology*; D010160:Palate, Soft / Q000502:physiology*; D010700:Phonetics; D013061:Speech Acoustics; D013065:Speech Intelligibility / Q000502:physiology; D013067:Speech Perception / Q000502:physiology; D013068:Speech Production Measurement / Q000379:methods; D014059:Tongue / Q000502:physiology*; D014584:User-Computer Interface