title: Decoding spatial attention by using cortical currents estimated from electroencephalography with near-infrared spectroscopy prior information.
abstract: For practical brain-machine interfaces (BMIs), electroencephalography (EEG) and near-infrared spectroscopy (NIRS) are the only current methods that are non-invasive and available in non-laboratory environments. However, the use of EEG and NIRS involves certain inherent problems. EEG signals are generally a mixture of neural activity from broad areas, some of which may not be related to the task targeted by BMI, hence impairing BMI performance. NIRS has an inherent time delay as it measures blood flow, which therefore detracts from practical real-time BMI utility. To try to improve real environment EEG-NIRS-based BMIs, we propose here a novel methodology in which the subjects' mental states are decoded from cortical currents estimated from EEG, with the help of information from NIRS. Using a Variational Bayesian Multimodal EncephaloGraphy (VBMEG) methodology, we incorporated a novel form of NIRS-based prior to capture event related desynchronization from isolated current sources on the cortical surface. Then, we applied a Bayesian logistic regression technique to decode subjects' mental states from further sparsified current sources. Applying our methodology to a spatial attention task, we found our EEG-NIRS-based decoder exhibited significant performance improvement over decoding methods based on EEG sensor signals alone. The advancement of our methodology, decoding from current sources sparsely isolated on the cortex, was also supported by neuroscientific considerations; intraparietal sulcus, a region known to be involved in spatial attention, was a key responsible region in our task. These results suggest that our methodology is not only a practical option for EEG-NIRS-based BMI applications, but also a potential tool to investigate brain activity in non-laboratory and naturalistic environments.
authors.affiliation: ATR Neural Information Analysis Laboratories, Kyoto 619-0288, Japan; Graduate School of Informatics, Kyoto University, Kyoto 611-0011, Japan.
authors.forename: Hiroshi
authors.initials: H
authors.lastname: Morioka
authors.affiliation: ATR Neural Information Analysis Laboratories, Kyoto 619-0288, Japan.
authors.forename: Atsunori
authors.initials: A
authors.lastname: Kanemura
authors.affiliation: ATR Neural Information Analysis Laboratories, Kyoto 619-0288, Japan.
authors.forename: Satoshi
authors.initials: S
authors.lastname: Morimoto
authors.affiliation: ATR Neural Information Analysis Laboratories, Kyoto 619-0288, Japan.
authors.forename: Taku
authors.initials: T
authors.lastname: Yoshioka
authors.affiliation: Graduate School of Informatics, Kyoto University, Kyoto 611-0011, Japan.
authors.forename: Shigeyuki
authors.initials: S
authors.lastname: Oba
authors.affiliation: ATR Neural Information Analysis Laboratories, Kyoto 619-0288, Japan.
authors.forename: Motoaki
authors.initials: M
authors.lastname: Kawanabe
authors.affiliation: ATR Neural Information Analysis Laboratories, Kyoto 619-0288, Japan; Graduate School of Informatics, Kyoto University, Kyoto 611-0011, Japan. Electronic address: ishii@i.kyoto-u.ac.jp.
authors.forename: Shin
authors.initials: S
authors.lastname: Ishii
mesh_terms: D000328:Adult; D001288:Attention / Q000502:physiology*; D001499:Bayes Theorem; D001921:Brain / Q000502:physiology*; D001931:Brain Mapping / Q000379:methods*; D062207:Brain-Computer Interfaces; D004569:Electroencephalography; D058407:Electroencephalography Phase Synchronization; D006801:Humans; D008297:Male; D012815:Signal Processing, Computer-Assisted; D013028:Space Perception / Q000502:physiology; D019265:Spectroscopy, Near-Infrared; D055815:Young Adult
keywords: Brain–machine interface (BMI); Electroencephalography (EEG); NIRS–EEG simultaneous measurement; Near-infrared spectroscopy (NIRS); Spatial attention; Variational Bayesian Multimodal EncephaloGraphy (VBMEG)