title: Decoding visual object categories from temporal correlations of ECoG signals.
abstract: How visual object categories are represented in the brain is one of the key questions in neuroscience. Studies on low-level visual features have shown that relative timings or phases of neural activity between multiple brain locations encode information. However, whether such temporal patterns of neural activity are used in the representation of visual objects is unknown. Here, we examined whether and how visual object categories could be predicted (or decoded) from temporal patterns of electrocorticographic (ECoG) signals from the temporal cortex in five patients with epilepsy. We used temporal correlations between electrodes as input features, and compared the decoding performance with features defined by spectral power and phase from individual electrodes. While using power or phase alone, the decoding accuracy was significantly better than chance, correlations alone or those combined with power outperformed other features. Decoding performance with correlations was degraded by shuffling the order of trials of the same category in each electrode, indicating that the relative time series between electrodes in each trial is critical. Analysis using a sliding time window revealed that decoding performance with correlations began to rise earlier than that with power. This earlier increase in performance was replicated by a model using phase differences to encode categories. These results suggest that activity patterns arising from interactions between multiple neuronal units carry additional information on visual object categories.
authors.affiliation: ATR Computational Neuroscience Laboratories, 2-2-2 Keihanna Science City, Kyoto 619-0288, Japan; Graduate School of Information Science, Nara Institute of Science and Technology, 8916-5 Takayama-cho, Ikoma, Nara 630-0192, Japan.
authors.forename: Kei
authors.initials: K
authors.lastname: Majima
authors.affiliation: Department of Physiology, Niigata University School of Medicine, 1-757 Asahi-machi St., Chuo-ku, Niigata 951-8510, Japan; Department of Neurosurgery, The University of Tokyo Graduate School of Medicine, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8654, Japan.
authors.forename: Takeshi
authors.initials: T
authors.lastname: Matsuo
authors.affiliation: Department of Physiology, Niigata University School of Medicine, 1-757 Asahi-machi St., Chuo-ku, Niigata 951-8510, Japan.
authors.forename: Keisuke
authors.initials: K
authors.lastname: Kawasaki
authors.affiliation: Department of Neurosurgery, The University of Tokyo Graduate School of Medicine, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8654, Japan.
authors.forename: Kensuke
authors.initials: K
authors.lastname: Kawai
authors.affiliation: Department of Neurosurgery, The University of Tokyo Graduate School of Medicine, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8654, Japan.
authors.forename: Nobuhito
authors.initials: N
authors.lastname: Saito
authors.affiliation: Department of Physiology, Niigata University School of Medicine, 1-757 Asahi-machi St., Chuo-ku, Niigata 951-8510, Japan; Center for Transdisciplinary Research, Niigata University, 1-757 Asahi-machi St., Chuo-ku, Niigata 951-8510, Japan.
authors.forename: Isao
authors.initials: I
authors.lastname: Hasegawa
authors.affiliation: ATR Computational Neuroscience Laboratories, 2-2-2 Keihanna Science City, Kyoto 619-0288, Japan; Graduate School of Information Science, Nara Institute of Science and Technology, 8916-5 Takayama-cho, Ikoma, Nara 630-0192, Japan. Electronic address: kmtn@atr.jp.
authors.forename: Yukiyasu
authors.initials: Y
authors.lastname: Kamitani
mesh_terms: D000328:Adult; D004569:Electroencephalography; D004827:Epilepsy / Q000503:physiopathology; D005260:Female; D006801:Humans; D008297:Male; D010775:Photic Stimulation; D012815:Signal Processing, Computer-Assisted; D013702:Temporal Lobe / Q000502:physiology*; D014796:Visual Perception / Q000502:physiology*; D055815:Young Adult
keywords: Decoding; ECoG; IT cortex; Object category; Temporal coding