title: Spontaneous Decoding of the Timing and Content of Human Object Perception from Cortical Surface Recordings Reveals Complementary Information in the Event-Related Potential and Broadband Spectral Change.
abstract: The link between object perception and neural activity in visual cortical areas is a problem of fundamental importance in neuroscience. Here we show that electrical potentials from the ventral temporal cortical surface in humans contain sufficient information for spontaneous and near-instantaneous identification of a subject's perceptual state. Electrocorticographic (ECoG) arrays were placed on the subtemporal cortical surface of seven epilepsy patients. Grayscale images of faces and houses were displayed rapidly in random sequence. We developed a template projection approach to decode the continuous ECoG data stream spontaneously, predicting the occurrence, timing and type of visual stimulus. In this setting, we evaluated the independent and joint use of two well-studied features of brain signals, broadband changes in the frequency power spectrum of the potential and deflections in the raw potential trace (event-related potential; ERP). Our ability to predict both the timing of stimulus onset and the type of image was best when we used a combination of both the broadband response and ERP, suggesting that they capture different and complementary aspects of the subject's perceptual state. Specifically, we were able to predict the timing and type of 96% of all stimuli, with less than 5% false positive rate and a ~20ms error in timing.
authors.affiliation: Departments of Neurosurgery, Stanford University, Stanford, California, United States of America.|NASA-Johnson Space Center, Houston, Texas, United States of America.|Program in Neurobiology and Behavior, University of Washington, Seattle, Washington, United States of America.
authors.forename: Kai J
authors.initials: KJ
authors.lastname: Miller
authors.affiliation: National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health, Albany, New York, United States of America.
authors.forename: Gerwin
authors.initials: G
authors.lastname: Schalk
authors.affiliation: Psychology, Stanford University, Stanford, California, United States of America.
authors.forename: Dora
authors.initials: D
authors.lastname: Hermes
authors.affiliation: Program in Neurobiology and Behavior, University of Washington, Seattle, Washington, United States of America.|Department of Neurological Surgery, University of Washington, Seattle, Washington, United States of America.|Center for Sensorimotor Neural Engineering, University of Washington, Seattle, Washington, United States of America.
authors.forename: Jeffrey G
authors.initials: JG
authors.lastname: Ojemann
authors.affiliation: Program in Neurobiology and Behavior, University of Washington, Seattle, Washington, United States of America.|Center for Sensorimotor Neural Engineering, University of Washington, Seattle, Washington, United States of America.|Computer Science and Engineering, University of Washington, Seattle, Washington, United States of America.
authors.forename: Rajesh P N
authors.initials: RP
authors.lastname: Rao
mesh_terms: D001931:Brain Mapping / Q000379:methods*; D019295:Computational Biology; D004569:Electroencephalography / Q000379:methods*; D004827:Epilepsy / Q000503:physiopathology; D005071:Evoked Potentials / Q000502:physiology*; D006801:Humans; D012815:Signal Processing, Computer-Assisted; D013702:Temporal Lobe / Q000502:physiology*; D014796:Visual Perception / Q000502:physiology*
references.citation: Vision Res. 2001 Jul;41(15):1891-9
references.pmid: 11412882
references.citation: J Neurosci. 2015 Jan 28;35(4):1513-20
references.pmid: 25632128
references.citation: PLoS Comput Biol. 2009 Dec;5(12):e1000609
references.pmid: 20019800
references.citation: Annu Int Conf IEEE Eng Med Biol Soc. 2010;2010:6015-8
references.pmid: 21097113
references.citation: J Neurosci. 2009 Mar 11;29(10):3132-7
references.pmid: 19279250
references.citation: PLoS Comput Biol. 2012;8(9):e1002655
references.pmid: 22969416
references.citation: Nat Commun. 2014 Dec 08;5:5672
references.pmid: 25482825
references.citation: J Neurophysiol. 2007 Jul;98(1):382-93
references.pmid: 17493920
references.citation: Cereb Cortex. 2015 Sep;25(9):2951-9
references.pmid: 24855114
references.citation: Cereb Cortex. 2011 Mar;21(3):616-24
references.pmid: 20624838
references.citation: Neuron. 2012 Oct 18;76(2):423-34
references.pmid: 23083743
references.citation: Neuroimage. 2005 Jul 1;26(3):839-51
references.pmid: 15955494
references.citation: Physiol Rev. 1985 Jan;65(1):37-100
references.pmid: 3880898
references.citation: Trends Cogn Sci. 2009 May;13(5):216-21
references.pmid: 19386535
references.citation: Neuroimage. 2014 Nov 1;101:215-24
references.pmid: 25019677
references.citation: Cereb Cortex. 1994 Sep-Oct;4(5):544-54
references.pmid: 7833655
references.citation: J Cogn Neurosci. 2001 Jan 1;13(1):90-101
references.pmid: 11224911
references.citation: J Vis. 2008 Sep 23;8(12):9.1-13
references.pmid: 18831622
references.citation: Neuropsychologia. 2016 Mar;83:14-28
references.pmid: 26212070
references.citation: Vision Res. 2002 Oct;42(23):2593-615
references.pmid: 12446033
references.citation: Neuroimage. 2012 Feb 1;59(3):2600-6
references.pmid: 21925278
references.citation: Nat Neurosci. 2014 Mar;17(3):455-62
references.pmid: 24464044
references.citation: Neuron. 1998 Aug;21(2):373-83
references.pmid: 9728918
references.citation: Cereb Cortex. 2004 Feb;14(2):165-73
references.pmid: 14704213
references.citation: Front Hum Neurosci. 2010 Oct 29;4:197
references.pmid: 21119778
references.citation: J Neurosci. 2009 Oct 28;29(43):13613-20
references.pmid: 19864573
references.citation: J Neurophysiol. 2017 Nov 1;118(5):2614-2627
references.pmid: 28814631
references.citation: Clin Neurophysiol. 2015 Nov;126(11):2150-61
references.pmid: 25680948
references.citation: Neuron. 2009 Apr 30;62(2):281-90
references.pmid: 19409272
references.citation: J Neurophysiol. 1994 Feb;71(2):821-5
references.pmid: 8176446
references.citation: Neuroimage. 1999 Feb;9(2):179-94
references.pmid: 9931268
references.citation: Nature. 1998 Apr 9;392(6676):598-601
references.pmid: 9560155
references.citation: Proc Biol Sci. 1994 Jul 22;257(1348):9-15
references.pmid: 8090795
references.citation: Science. 2014 Feb 28;343(6174):1006-10
references.pmid: 24482117
references.citation: Conscious Cogn. 2004 Dec;13(4):657-90
references.pmid: 15522626
references.citation: J Neurophysiol. 2007 Jun;97(6):4296-309
references.pmid: 17428910
references.citation: Neuroimage. 2014 Jan 15;85 Pt 2:711-20
references.pmid: 24018305
references.citation: J Neurosci. 1997 Jun 1;17(11):4302-11
references.pmid: 9151747
references.citation: IEEE Trans Biomed Eng. 2004 Jun;51(6):1034-43
references.pmid: 15188875
references.citation: Nat Neurosci. 2000 Sep;3(9):946-53
references.pmid: 10966627
references.citation: Neuroimage. 2013 Dec;83:1063-73
references.pmid: 23927900
references.citation: Science. 2005 Nov 4;310(5749):863-6
references.pmid: 16272124
references.citation: Cereb Cortex. 2011 May;21(5):1213-21
references.pmid: 20961973
references.citation: Science. 1967 Dec 29;158(3809):1704-5
references.pmid: 6059654
references.citation: PLoS One. 2010 Dec 30;5(12):e14465
references.pmid: 21209937
references.citation: J Neurosci Methods. 2010 Jan 15;185(2):293-8
references.pmid: 19836416
references.citation: Med Image Anal. 1996 Mar;1(1):35-51
references.pmid: 9873920
references.citation: Vision Res. 2006 May;46(11):1762-76
references.pmid: 16289663
references.citation: J Vis. 2010 Apr 28;10(4):16.1-17
references.pmid: 20465335
references.citation: J Neurophysiol. 1995 Sep;74(3):1192-9
references.pmid: 7500143
references.citation: J Neurophysiol. 2015 Jul;114(1):256-63
references.pmid: 25972581
references.citation: Front Hum Neurosci. 2010 Nov 11;4:195
references.pmid: 21267419