title: High-Resolution, Non-Invasive Imaging of Upper Vocal Tract Articulators Compatible with Human Brain Recordings.
abstract: A complete neurobiological understanding of speech motor control requires determination of the relationship between simultaneously recorded neural activity and the kinematics of the lips, jaw, tongue, and larynx. Many speech articulators are internal to the vocal tract, and therefore simultaneously tracking the kinematics of all articulators is nontrivial--especially in the context of human electrophysiology recordings. Here, we describe a noninvasive, multi-modal imaging system to monitor vocal tract kinematics, demonstrate this system in six speakers during production of nine American English vowels, and provide new analysis of such data. Classification and regression analysis revealed considerable variability in the articulator-to-acoustic relationship across speakers. Non-negative matrix factorization extracted basis sets capturing vocal tract shapes allowing for higher vowel classification accuracy than traditional methods. Statistical speech synthesis generated speech from vocal tract measurements, and we demonstrate perceptual identification. We demonstrate the capacity to predict lip kinematics from ventral sensorimotor cortical activity. These results demonstrate a multi-modal system to non-invasively monitor articulator kinematics during speech production, describe novel analytic methods for relating kinematic data to speech acoustics, and provide the first decoding of speech kinematics from electrocorticography. These advances will be critical for understanding the cortical basis of speech production and the creation of vocal prosthetics.
authors.affiliation: Biological Systems and Engineering Division & Computational Research Division, Lawrence Berkeley National Laboratories (LBNL), Berkeley, California, United States of America.|Department of Neurological Surgery, University of California San Francisco (UCSF), San Francisco, California, United States of America.
authors.forename: Kristofer E
authors.initials: KE
authors.lastname: Bouchard
authors.affiliation: Department of Neurological Surgery, University of California San Francisco (UCSF), San Francisco, California, United States of America.|Center for Integrative Neuroscience, UCSF, San Francisco, California, United States of America.
authors.forename: David F
authors.initials: DF
authors.lastname: Conant
authors.affiliation: Department of Neurological Surgery, University of California San Francisco (UCSF), San Francisco, California, United States of America.|Center for Integrative Neuroscience, UCSF, San Francisco, California, United States of America.
authors.forename: Gopala K
authors.initials: GK
authors.lastname: Anumanchipalli
authors.affiliation: Department of Neurological Surgery, University of California San Francisco (UCSF), San Francisco, California, United States of America.|Center for Integrative Neuroscience, UCSF, San Francisco, California, United States of America.
authors.forename: Benjamin
authors.initials: B
authors.lastname: Dichter
authors.affiliation: Department of Neurological Surgery, University of California San Francisco (UCSF), San Francisco, California, United States of America.|Center for Integrative Neuroscience, UCSF, San Francisco, California, United States of America.
authors.forename: Kris S
authors.initials: KS
authors.lastname: Chaisanguanthum
authors.affiliation: Department of Linguistics, University of California (UCB), Berkeley, California, United States of America.
authors.forename: Keith
authors.initials: K
authors.lastname: Johnson
authors.affiliation: Department of Neurological Surgery, University of California San Francisco (UCSF), San Francisco, California, United States of America.|Center for Integrative Neuroscience, UCSF, San Francisco, California, United States of America.
authors.forename: Edward F
authors.initials: EF
authors.lastname: Chang
mesh_terms: D000465:Algorithms; D001696:Biomechanical Phenomena; D001921:Brain / Q000033:anatomy & histology / Q000502:physiology*; D001931:Brain Mapping; D003952:Diagnostic Imaging / Q000379:methods*; D000069280:Electrocorticography; D005260:Female; D006801:Humans; D007568:Jaw / Q000033:anatomy & histology / Q000294:innervation / Q000502:physiology; D007830:Larynx / Q000502:physiology; D008046:Lip / Q000033:anatomy & histology / Q000294:innervation / Q000502:physiology; D008297:Male; D008959:Models, Neurological; D010700:Phonetics; D013003:Somatosensory Cortex / Q000033:anatomy & histology / Q000502:physiology; D013061:Speech Acoustics; D013068:Speech Production Measurement / Q000379:methods; D014059:Tongue / Q000033:anatomy & histology / Q000294:innervation / Q000502:physiology; D014827:Vocal Cords / Q000033:anatomy & histology / Q000294:innervation / Q000502:physiology*