title: Rare neural correlations implement robotic conditioning with delayed rewards and disturbances.
abstract: Neural conditioning associates cues and actions with following rewards. The environments in which robots operate, however, are pervaded by a variety of disturbing stimuli and uncertain timing. In particular, variable reward delays make it difficult to reconstruct which previous actions are responsible for following rewards. Such an uncertainty is handled by biological neural networks, but represents a challenge for computational models, suggesting the lack of a satisfactory theory for robotic neural conditioning. The present study demonstrates the use of rare neural correlations in making correct associations between rewards and previous cues or actions. Rare correlations are functional in selecting sparse synapses to be eligible for later weight updates if a reward occurs. The repetition of this process singles out the associating and reward-triggering pathways, and thereby copes with distal rewards. The neural network displays macro-level classical and operant conditioning, which is demonstrated in an interactive real-life human-robot interaction. The proposed mechanism models realistic conditioning in humans and animals and implements similar behaviors in neuro-robotic platforms.
authors.affiliation: Faculty of Technology, Research Institute for Cognition and Robotics (CoR-Lab), Bielefeld University Bielefeld, Germany.
authors.forename: Andrea
authors.initials: A
authors.lastname: Soltoggio
authors.forename: Andre
authors.initials: A
authors.lastname: Lemme
authors.forename: Felix
authors.initials: F
authors.lastname: Reinhart
authors.forename: Jochen J
authors.initials: JJ
authors.lastname: Steil
keywords: classical conditioning; distal reward; instrumental conditioning; neuromodulation; robotics
references.citation: Neural Netw. 2002 Jun-Jul;15(4-6):479-93
references.pmid: 12371506
references.citation: Neural Netw. 2002 Jun-Jul;15(4-6):761-74
references.pmid: 12371525
references.citation: Nature. 1997 Feb 6;385(6616):533-6
references.pmid: 9020359
references.citation: J Exp Psychol. 1948 Apr;38(2):168-72
references.pmid: 18913665
references.citation: Nat Neurosci. 2000 Dec;3(12):1266-73
references.pmid: 11100147
references.citation: Neural Comput. 2007 Oct;19(10):2694-719
references.pmid: 17716008
references.citation: J Neurosci. 1993 Mar;13(3):900-13
references.pmid: 8441015
references.citation: J Comp Neurol. 2005 Dec 5;493(1):99-110
references.pmid: 16254995
references.citation: Annu Rev Psychol. 2004;55:235-69
references.pmid: 14744216
references.citation: Science. 2002 May 31;296(5573):1706-9
references.pmid: 12040200
references.citation: Science. 1997 Mar 14;275(5306):1593-9
references.pmid: 9054347
references.citation: Neural Comput. 2013 Apr;25(4):940-78
references.pmid: 23339615
references.citation: PLoS Comput Biol. 2008 Oct;4(10):e1000180
references.pmid: 18846203
references.citation: Science. 1973 Aug 17;181(4100):682-4
references.pmid: 4724483
references.citation: Front Comput Neurosci. 2012 Jan 31;6:5
references.pmid: 22319488
references.citation: Neural Netw. 2012 Oct;34:28-41
references.pmid: 22796669
references.citation: Nat Rev Neurosci. 2011 Jan;12(1):17-30
references.pmid: 21170072
references.citation: Neural Comput. 2007 Jun;19(6):1468-502
references.pmid: 17444757
references.citation: J Physiol. 1965 Nov;181(1):1-27
references.pmid: 5866283
references.citation: Neural Comput. 2009 Feb;21(2):301-39
references.pmid: 19196231
references.citation: Proc Natl Acad Sci U S A. 2000 Oct 10;97(21):11581-6
references.pmid: 11027355
references.citation: J Neurosci. 1996 Mar 1;16(5):1936-47
references.pmid: 8774460
references.citation: Neural Comput. 1998 May 15;10(4):771-805
references.pmid: 9573404
references.citation: Behav Brain Res. 1995 Feb;67(1):1-27
references.pmid: 7748496
references.citation: Neural Netw. 2002 Jun-Jul;15(4-6):495-506
references.pmid: 12371507
references.citation: Neural Netw. 2002 Jun-Jul;15(4-6):507-21
references.pmid: 12371508
references.citation: J Neurophysiol. 2007 Dec;98(6):3648-65
references.pmid: 17928565
references.citation: PLoS Comput Biol. 2009 Dec;5(12):e1000586
references.pmid: 19997492
references.citation: Learn Mem. 2010 Dec 22;18(1):58-70
references.pmid: 21191043
references.citation: Nature. 1993 Nov 4;366:59-63
references.pmid: 24308080
references.citation: Neural Comput. 2010 Jun;22(6):1399-444
references.pmid: 20141476
references.citation: Trends Cogn Sci. 1999 Sep;3(9):351-359
references.pmid: 10461198
references.citation: Annu Rev Neurosci. 1996;19:379-404
references.pmid: 8833448
references.citation: PLoS Comput Biol. 2011 May;7(5):e1001133
references.pmid: 21589888
references.citation: J Neurosci. 2008 Jan 2;28(1):133-42
references.pmid: 18171931
references.citation: Cereb Cortex. 2007 Oct;17(10):2443-52
references.pmid: 17220510
references.citation: Neural Netw. 1999 Oct;12(7-8):961-974
references.pmid: 12662639
references.citation: J Neurosci. 1981 Dec;1(12):1426-37
references.pmid: 7320755