title: Word pair classification during imagined speech using direct brain recordings.
abstract: People that cannot communicate due to neurological disorders would benefit from an internal speech decoder. Here, we showed the ability to classify individual words during imagined speech from electrocorticographic signals. In a word imagery task, we used high gamma (70-150 Hz) time features with a support vector machine model to classify individual words from a pair of words. To account for temporal irregularities during speech production, we introduced a non-linear time alignment into the SVM kernel. Classification accuracy reached 88% in a two-class classification framework (50% chance level), and average classification accuracy across fifteen word-pairs was significant across five subjects (mean = 58%; p < 0.05). We also compared classification accuracy between imagined speech, overt speech and listening. As predicted, higher classification accuracy was obtained in the listening and overt speech conditions (mean = 89% and 86%, respectively; p < 0.0001), where speech stimuli were directly presented. The results provide evidence for a neural representation for imagined words in the temporal lobe, frontal lobe and sensorimotor cortex, consistent with previous findings in speech perception and production. These data represent a proof of concept study for basic decoding of speech imagery, and delineate a number of key challenges to usage of speech imagery neural representations for clinical applications.
authors.affiliation: Defitech Chair in Brain-Machine Interface, Center for Neuroprosthetics, Ecole Polytechnique Fédérale de Lausanne, Switzerland.|Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA.
authors.forename: Stephanie
authors.initials: S
authors.lastname: Martin
authors.affiliation: National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health, Albany, NY, USA.|Department of Neurology, Albany Medical College, Albany, NY, USA.
authors.forename: Peter
authors.initials: P
authors.lastname: Brunner
authors.affiliation: Defitech Chair in Brain-Machine Interface, Center for Neuroprosthetics, Ecole Polytechnique Fédérale de Lausanne, Switzerland.
authors.forename: Iñaki
authors.initials: I
authors.lastname: Iturrate
authors.affiliation: Defitech Chair in Brain-Machine Interface, Center for Neuroprosthetics, Ecole Polytechnique Fédérale de Lausanne, Switzerland.
authors.forename: José Del R
authors.initials: Jdel R
authors.lastname: Millán
authors.affiliation: National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health, Albany, NY, USA.|Department of Neurology, Albany Medical College, Albany, NY, USA.
authors.forename: Gerwin
authors.initials: G
authors.lastname: Schalk
authors.affiliation: Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA.|Department of Psychology, University of California, Berkeley, CA, USA.
authors.forename: Robert T
authors.initials: RT
authors.lastname: Knight
authors.affiliation: Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA.
authors.forename: Brian N
authors.initials: BN
authors.lastname: Pasley
mesh_terms: D000161:Acoustic Stimulation; D001307:Auditory Perception / Q000502:physiology; D001921:Brain / Q000502:physiology*; D001931:Brain Mapping; D004192:Discrimination, Psychological; D004566:Electrodes; D004569:Electroencephalography; D065818:Gamma Rhythm / Q000502:physiology; D006801:Humans; D007092:Imagination; D012372:ROC Curve; D013060:Speech; D013997:Time Factors; D014825:Vocabulary