title: Restoring speech following total removal of the larynx by a learned transformation from sensor data to acoustics.
abstract: Total removal of the larynx may be required to treat laryngeal cancer: speech is lost. This article shows that it may be possible to restore speech by sensing movement of the remaining speech articulators and use machine learning algorithms to derive a transformation to convert this sensor data into an acoustic signal. The resulting "silent speech," which may be delivered in real time, is intelligible and sounds natural. The identity of the speaker is recognisable. The sensing technique involves attaching small, unobtrusive magnets to the lips and tongue and monitoring changes in the magnetic field induced by their movement.
authors.affiliation: School of Engineering, University of Hull, Kingston upon Hull, United Kingdom J.M.Gilbert@Hull.ac.uk.
authors.forename: James M
authors.initials: JM
authors.lastname: Gilbert
authors.affiliation: Department of Computer Science, University of Sheffield, Sheffield, United Kingdom J.Gonzalez@Sheffield.ac.uk.
authors.forename: Jose A
authors.initials: JA
authors.lastname: Gonzalez
authors.affiliation: School of Engineering, University of Hull, Kingston upon Hull, United Kingdom L.Cheah@Hull.ac.uk.
authors.forename: Lam A
authors.initials: LA
authors.lastname: Cheah
authors.affiliation: Hull and East Yorkshire Hospitals Trust, Castle Hill Hospital, Cottingham, United Kingdom srell@doctors.org.uk.
authors.forename: Stephen R
authors.initials: SR
authors.lastname: Ell
authors.affiliation: Department of Computer Science, University of Sheffield, Sheffield, United Kingdom P.Green@Sheffield.ac.uk, R.K.Moore@Sheffield.ac.uk.
authors.forename: Phil
authors.initials: P
authors.lastname: Green
authors.affiliation: Department of Computer Science, University of Sheffield, Sheffield, United Kingdom P.Green@Sheffield.ac.uk, R.K.Moore@Sheffield.ac.uk.
authors.forename: Roger K
authors.initials: RK
authors.lastname: Moore
authors.affiliation: Practical Control Limited, Sheffield, United Kingdom Ed.holdsworth@practicalcontrol.com.
authors.forename: Ed
authors.initials: E
authors.lastname: Holdsworth
mesh_terms: D000162:Acoustics / Q000295:instrumentation*; D001696:Biomechanical Phenomena; D006801:Humans; D007825:Laryngectomy; D007831:Larynx, Artificial; D008046:Lip / Q000502:physiology*; D000069550:Machine Learning; D060526:Magnetic Fields; D008280:Magnetics / Q000295:instrumentation*; D059346:Magnets; D011474:Prosthesis Design; D020127:Recovery of Function; D012815:Signal Processing, Computer-Assisted; D013018:Sound Spectrography; D013061:Speech Acoustics; D013065:Speech Intelligibility; D013997:Time Factors; D014059:Tongue / Q000502:physiology*; D014159:Transducers; D014833:Voice Quality