title: Real-time parallel processing of grammatical structure in the fronto-striatal system: a recurrent network simulation study using reservoir computing.
abstract: Sentence processing takes place in real-time. Previous words in the sentence can influence the processing of the current word in the timescale of hundreds of milliseconds. Recent neurophysiological studies in humans suggest that the fronto-striatal system (frontal cortex, and striatum--the major input locus of the basal ganglia) plays a crucial role in this process. The current research provides a possible explanation of how certain aspects of this real-time processing can occur, based on the dynamics of recurrent cortical networks, and plasticity in the cortico-striatal system. We simulate prefrontal area BA47 as a recurrent network that receives on-line input about word categories during sentence processing, with plastic connections between cortex and striatum. We exploit the homology between the cortico-striatal system and reservoir computing, where recurrent frontal cortical networks are the reservoir, and plastic cortico-striatal synapses are the readout. The system is trained on sentence-meaning pairs, where meaning is coded as activation in the striatum corresponding to the roles that different nouns and verbs play in the sentences. The model learns an extended set of grammatical constructions, and demonstrates the ability to generalize to novel constructions. It demonstrates how early in the sentence, a parallel set of predictions are made concerning the meaning, which are then confirmed or updated as the processing of the input sentence proceeds. It demonstrates how on-line responses to words are influenced by previous words in the sentence, and by previous sentences in the discourse, providing new insight into the neurophysiology of the P600 ERP scalp response to grammatical complexity. This demonstrates that a recurrent neural network can decode grammatical structure from sentences in real-time in order to generate a predictive representation of the meaning of the sentences. This can provide insight into the underlying mechanisms of human cortico-striatal function in sentence processing.
authors: INSERM U846 Stem Cell and Brain Research Institute, Bron Cedex, France.
authors: Xavier
authors: X
authors: Hinaut
authors: Peter Ford
authors: PF
authors: Dominey
mesh_terms: D001185:Artificial Intelligence; D003198:Computer Simulation; D003342:Corpus Striatum / Q000502:physiology*; D005625:Frontal Lobe / Q000502:physiology*; D006801:Humans; D016571:Neural Networks, Computer; D012660:Semantics; D013067:Speech Perception / Q000502:physiology*
references: Neuroimage. 2001 Jan;13(1):110-8
references: 11133314
references: Brain Lang. 1991 Aug;41(2):123-48
references: 1933255
references: J Psycholinguist Res. 2000 Mar;29(2):231-40
references: 10709187
references: Cognition. 2004 May-Jun;92(1-2):231-70
references: 15037131
references: Philos Trans R Soc Lond B Biol Sci. 2005 Apr 29;360(1456):815-36
references: 15937014
references: Neuropsychologia. 2000;38(11):1531-49
references: 10906378
references: J Psycholinguist Res. 2000 Mar;29(2):241-8
references: 10709188
references: J Cogn Neurosci. 1995 Summer;7(3):311-36
references: 23961864
references: J Int Neuropsychol Soc. 2003 Nov;9(7):1053-60
references: 14738286
references: J Comp Neurol. 1981 Jun 20;199(2):205-19
references: 7251940
references: Brain Res Cogn Brain Res. 2001 Apr;11(2):305-23
references: 11275491
references: Neuroreport. 2008 May 28;19(8):811-5
references: 18463492
references: Eur J Neurosci. 2001 Mar;13(6):1071-7
references: 11285003
references: Cortex. 2009 Sep;45(8):991-1011
references: 19473654
references: Neuroimage. 2003 Nov;20 Suppl 1:S8-17
references: 14597292
references: J Child Lang. 2003 May;30(2):333-70
references: 12846301
references: J Cogn Neurosci. 2003 Aug 15;15(6):883-99
references: 14511541
references: Behav Brain Sci. 2006 Feb;29(1):37-70; discussion 70-108
references: 16542539
references: Biol Cybern. 1995 Aug;73(3):265-74
references: 7548314
references: Cortex. 2009 Sep;45(8):1012-8
references: 19446801
references: Neuropsychology. 2003 Jan;17(1):133-42
references: 12597082
references: Psychol Sci. 2011 Jun;22(6):829-34
references: 21586764
references: Cognition. 1998 Aug;68(1):1-76
references: 9775516
references: Neural Netw. 2012 Nov;35:1-9
references: 22885243
references: Clin Neurophysiol. 2003 Feb;114(2):336-40
references: 12559242
references: Neural Comput. 2010 May;22(5):1272-311
references: 20028227
references: J Cogn Neurosci. 2007 Jul;19(7):1193-205
references: 17583994
references: Brain Lang. 2003 Aug;86(2):207-25
references: 12921765
references: J Physiol Paris. 2011 Jan-Jun;105(1-3):16-24
references: 21939760
references: Philos Trans R Soc Lond B Biol Sci. 2007 May 29;362(1481):801-11
references: 17412680
references: Neural Netw. 2007 Apr;20(3):391-403
references: 17517492
references: Cognition. 1982 May;11(3):245-99
references: 7199413
references: Neural Netw. 2007 Apr;20(3):424-32
references: 17556116
references: Nat Rev Neurosci. 2001 Oct;2(10):717-26
references: 11584309
references: IEEE Trans Neural Netw. 2008 Jun;19(6):983-95
references: 18541499
references: Neural Comput. 2002 Nov;14(11):2531-60
references: 12433288
references: Science. 1980 Jan 11;207(4427):203-5
references: 7350657
references: J Cogn Neurosci. 2006 Dec;18(12):2088-107
references: 17129193
references: Behav Brain Res. 2009 Apr 12;199(1):108-18
references: 18948145
references: Philos Trans R Soc Lond B Biol Sci. 2008 Mar 12;363(1493):1055-69
references: 17890190
references: J Exp Psychol Learn Mem Cogn. 2005 May;31(3):443-67
references: 15910130
references: Cognition. 1985 Nov;21(2):117-75
references: 2419022
references: J Neurosci. 2012 Mar 14;32(11):3665-78
references: 22423089
references: J Comp Neurol. 1989 Apr 1;282(1):80-97
references: 2468699
references: Neural Netw. 2007 Apr;20(3):335-52
references: 17517495
references: Cognition. 2002 Oct;85(3):B83-92
references: 12169415
references: Proc Natl Acad Sci U S A. 2006 Feb 14;103(7):2458-63
references: 16461904
references: Science. 2004 Apr 2;304(5667):78-80
references: 15064413
references: Trends Neurosci. 1990 Jul;13(7):277-80
references: 1695403
references: PLoS One. 2009 Jul 28;4(7):e6393
references: 19636432
references: Brain Lang. 2006 Jun;97(3):243-57
references: 16332387
references: Cogn Neurodyn. 2008 Sep;2(3):229-55
references: 19003488
references: J Mem Lang. 2007 Oct 1;57(3):348-379
references: 19668599
references: Cortex. 2006 May;42(4):476-9
references: 16881253
references: Brain Lang. 2009 May-Jun;109(2-3):80-92
references: 18835637
references: Neural Netw. 2005 Sep;18(7):878-95
references: 15936173
references: J Psycholinguist Res. 2000 Mar;29(2):217-29
references: 10709186
references: Brain Res Cogn Brain Res. 2003 Jan;15(2):165-77
references: 12429368
references: Cereb Cortex. 2002 Oct;12(10):1040-7
references: 12217967