title: A procedure for estimating gestural scores from speech acoustics.
abstract: Speech can be represented as a constellation of constricting vocal tract actions called gestures, whose temporal patterning with respect to one another is expressed in a gestural score. Current speech datasets do not come with gestural annotation and no formal gestural annotation procedure exists at present. This paper describes an iterative analysis-by-synthesis landmark-based time-warping architecture to perform gestural annotation of natural speech. For a given utterance, the Haskins Laboratories Task Dynamics and Application (TADA) model is employed to generate a corresponding prototype gestural score. The gestural score is temporally optimized through an iterative timing-warping process such that the acoustic distance between the original and TADA-synthesized speech is minimized. This paper demonstrates that the proposed iterative approach is superior to conventional acoustically-referenced dynamic timing-warping procedures and provides reliable gestural annotation for speech datasets.
authors.affiliation: Haskins Laboratories, 300 George Street, Suite 900, New Haven, Connecticut 06511, USA. nam@haskins.yale.edu
authors.forename: Hosung
authors.initials: H
authors.lastname: Nam
authors.forename: Vikramjit
authors.initials: V
authors.lastname: Mitra
authors.forename: Mark
authors.initials: M
authors.lastname: Tiede
authors.forename: Mark
authors.initials: M
authors.lastname: Hasegawa-Johnson
authors.forename: Carol
authors.initials: C
authors.lastname: Espy-Wilson
authors.forename: Elliot
authors.initials: E
authors.lastname: Saltzman
authors.forename: Louis
authors.initials: L
authors.lastname: Goldstein
mesh_terms: D000162:Acoustics; D001696:Biomechanical Phenomena; D005260:Female; D005868:Gestures; D005931:Glottis / Q000502:physiology*; D006801:Humans; D008297:Male; D008962:Models, Theoretical; D009055:Mouth / Q000502:physiology*; D012815:Signal Processing, Computer-Assisted; D013018:Sound Spectrography; D013061:Speech Acoustics; D013068:Speech Production Measurement / Q000379:methods; D013997:Time Factors; D014833:Voice Quality