title: Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules.
abstract: Classical Hebbian learning puts the emphasis on joint pre- and postsynaptic activity, but neglects the potential role of neuromodulators. Since neuromodulators convey information about novelty or reward, the influence of neuromodulators on synaptic plasticity is useful not just for action learning in classical conditioning, but also to decide "when" to create new memories in response to a flow of sensory stimuli. In this review, we focus on timing requirements for pre- and postsynaptic activity in conjunction with one or several phasic neuromodulatory signals. While the emphasis of the text is on conceptual models and mathematical theories, we also discuss some experimental evidence for neuromodulation of Spike-Timing-Dependent Plasticity. We highlight the importance of synaptic mechanisms in bridging the temporal gap between sensory stimulation and neuromodulatory signals, and develop a framework for a class of neo-Hebbian three-factor learning rules that depend on presynaptic activity, postsynaptic variables as well as the influence of neuromodulators.
authors.affiliation: School of Computer Science and Brain Mind Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne Lausanne, Switzerland.
authors.forename: Nicolas
authors.initials: N
authors.lastname: Frémaux
authors.affiliation: School of Computer Science and Brain Mind Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne Lausanne, Switzerland.
authors.forename: Wulfram
authors.initials: W
authors.lastname: Gerstner
mesh_terms: D000200:Action Potentials / Q000502:physiology*; D000818:Animals; D006801:Humans; D007858:Learning / Q000502:physiology*; D008959:Models, Neurological; D009473:Neuronal Plasticity / Q000502:physiology*; D009474:Neurons / Q000502:physiology*
keywords: STDP; neuromodulation; novelty; plasticity; reward learning; spiking neuron networks; synaptic plasticity (LTP/LTD)
references.citation: Science. 1997 Mar 14;275(5306):1593-9
references.pmid: 9054347
references.citation: Neurosci Lett. 1992 Aug 3;142(1):95-9
references.pmid: 1357611
references.citation: Proc Natl Acad Sci U S A. 2009 Aug 4;106(31):13028-33
references.pmid: 19620735
references.citation: Nat Rev Neurosci. 2009 Feb;10(2):113-25
references.pmid: 19145235
references.citation: J Comp Neurol. 2006 Nov 1;499(1):49-63
references.pmid: 16958109
references.citation: Neural Comput. 2009 Feb;21(2):301-39
references.pmid: 19196231
references.citation: Neuron. 2014 Oct 22;84(2):486-96
references.pmid: 25284006
references.citation: Neural Comput. 2006 Jun;18(6):1318-48
references.pmid: 16764506
references.citation: Nat Neurosci. 2007 Apr;10(4):485-93
references.pmid: 17351638
references.citation: Proc Natl Acad Sci U S A. 2006 Oct 10;103(41):15224-9
references.pmid: 17008410
references.citation: Curr Opin Neurobiol. 2014 Apr;25:156-63
references.pmid: 24509098
references.citation: Neuroscience. 1984 Dec;13(4):1189-215
references.pmid: 6152036
references.citation: Neuropharmacology. 2007 Jan;52(1):24-40
references.pmid: 16919684
references.citation: Vision Res. 2012 May 15;61:95-9
references.pmid: 22119774
references.citation: Trends Neurosci. 2007 May;30(5):211-9
references.pmid: 17367873
references.citation: Nature. 1997 Feb 6;385(6616):533-6
references.pmid: 9020359
references.citation: Annu Rev Neurosci. 2005;28:403-50
references.pmid: 16022602
references.citation: Nat Neurosci. 2009 Mar;12(3):250-2
references.pmid: 19219040
references.citation: Annu Rev Neurosci. 2008;31:25-46
references.pmid: 18275283
references.citation: J Neurosci. 2012 May 9;32(19):6611-20
references.pmid: 22573683
references.citation: PLoS Comput Biol. 2011 Jun;7(6):e1002092
references.pmid: 21738460
references.citation: Neuron. 2012 Oct 4;76(1):1-11
references.pmid: 23040802
references.citation: Biol Cybern. 2000 Sep;83(3):287-99
references.pmid: 11007302
references.citation: Front Synaptic Neurosci. 2010 Jul 02;2:23
references.pmid: 21423509
references.citation: Nature. 2002 Jul 18;418(6895):326-31
references.pmid: 12124625
references.citation: Neural Comput. 2007 Jun;19(6):1468-502
references.pmid: 17444757
references.citation: Front Syst Neurosci. 2014 Sep 18;8:172
references.pmid: 25278848
references.citation: J Neurophysiol. 1998 Jul;80(1):1-27
references.pmid: 9658025
references.citation: Annu Rev Neurosci. 2000;23:185-215
references.pmid: 10845063
references.citation: Philos Trans R Soc Lond B Biol Sci. 2003 Apr 29;358(1432):829-42
references.pmid: 12740130
references.citation: Physiol Rev. 2008 Apr;88(2):769-840
references.pmid: 18391179
references.citation: Neuron. 2011 Jul 14;71(1):155-65
references.pmid: 21745645
references.citation: Neural Comput. 2001 Apr;13(4):841-62
references.pmid: 11255572
references.citation: Nat Rev Neurosci. 2000 Oct;1(1):11-20
references.pmid: 11252764
references.citation: Nat Neurosci. 2012 Jul;15(7):1040-6
references.pmid: 22660479
references.citation: Nature. 2011 Feb 3;470(7332):101-4
references.pmid: 21228773
references.citation: Neural Comput. 2007 Mar;19(3):639-71
references.pmid: 17298228
references.citation: Trends Neurosci. 1993 Nov;16(11):480-7
references.pmid: 7507622
references.citation: Neuron. 2007 Nov 21;56(4):701-13
references.pmid: 18031686
references.citation: PLoS Comput Biol. 2013 Apr;9(4):e1003024
references.pmid: 23592970
references.citation: Neural Netw. 2002 Jun-Jul;15(4-6):507-21
references.pmid: 12371508
references.citation: Nature. 2012 Feb 2;482(7383):47-52
references.pmid: 22278062
references.citation: Proc Natl Acad Sci U S A. 1997 Feb 18;94(4):1494-9
references.pmid: 9037081
references.citation: Nature. 1977 Apr 21;266(5604):737-9
references.pmid: 195211
references.citation: J Neurophysiol. 2004 May;91(5):2259-72
references.pmid: 14736863
references.citation: Neuron. 2007 Apr 5;54(1):73-87
references.pmid: 17408579
references.citation: Front Synaptic Neurosci. 2011 Aug 29;3:4
references.pmid: 22007168
references.citation: PLoS Comput Biol. 2008 Dec;4(12):e1000248
references.pmid: 19112486
references.citation: Annu Rev Neurosci. 2000;23:649-711
references.pmid: 10845078
references.citation: Science. 1998 Mar 13;279(5357):1714-8
references.pmid: 9497289
references.citation: Biol Cybern. 2008 Jun;98(6):459-78
references.pmid: 18491160
references.citation: Cereb Cortex. 2007 Oct;17(10):2443-52
references.pmid: 17220510
references.citation: Science. 1999 Sep 17;285(5435):1870-4
references.pmid: 10489359
references.citation: Neuron. 2014 Jun 18;82(6):1394-406
references.pmid: 24945778
references.citation: J Neurosci. 2011 Feb 23;31(8):2843-54
references.pmid: 21414906
references.citation: Front Comput Neurosci. 2014 Apr 04;8:38
references.pmid: 24772078
references.citation: PLoS Comput Biol. 2008 Mar;4(3):e1000007
references.pmid: 18369414
references.citation: Prog Neurobiol. 2011 Jun;94(1):64-76
references.pmid: 21435370
references.citation: Nat Rev Neurosci. 2011 Jan;12(1):17-30
references.pmid: 21170072
references.citation: Biol Cybern. 2002 Dec;87(5-6):459-70
references.pmid: 12461635
references.citation: Nature. 1993 Jan 7;361(6407):31-9
references.pmid: 8421494
references.citation: Nature. 2001 Jul 5;412(6842):43-8
references.pmid: 11452299
references.citation: PLoS Comput Biol. 2013 Apr;9(4):e1003037
references.pmid: 23633941
references.citation: Behav Brain Res. 2011 Aug 10;221(2):505-14
references.pmid: 21130117
references.citation: Neuron. 2002 Oct 10;36(2):241-63
references.pmid: 12383780
references.citation: Neuron. 2005 May 19;46(4):681-92
references.pmid: 15944135
references.citation: Nat Neurosci. 2013 Jul;16(7):966-73
references.pmid: 23708143
references.citation: Neuron. 2010 Dec 9;68(5):948-63
references.pmid: 21145007
references.citation: Neuroscience. 1999;91(3):871-90
references.pmid: 10391468
references.citation: Nat Neurosci. 2010 Mar;13(3):344-52
references.pmid: 20098420
references.citation: Nat Neurosci. 2013 Jul;16(7):925-33
references.pmid: 23708144
references.citation: Nat Rev Neurosci. 2009 Mar;10(3):211-23
references.pmid: 19190638
references.citation: J Neurosci. 2007 Jul 11;27(28):7476-81
references.pmid: 17626208
references.citation: Neuropharmacology. 2000 Jun 8;39(8):1337-56
references.pmid: 10818251
references.citation: Network. 2006 Sep;17(3):235-52
references.pmid: 17162613
references.citation: Front Synaptic Neurosci. 2010 Oct 25;2:146
references.pmid: 21423532
references.citation: Neuron. 2003 Dec 18;40(6):1063-73
references.pmid: 14687542
references.citation: Res Publ Assoc Res Nerv Ment Dis. 1972;50:376-89
references.pmid: 4341953
references.citation: PLoS Comput Biol. 2011 May;7(5):e1001133
references.pmid: 21589888
references.citation: Psychol Rev. 2009 Jul;116(3):540-66
references.pmid: 19618986
references.citation: Proc Natl Acad Sci U S A. 2010 Sep 14;107(37):16366-71
references.pmid: 20805489
references.citation: Nat Rev Neurosci. 2006 Dec;7(12):967-75
references.pmid: 17115078
references.citation: Hippocampus. 2000;10(1):1-16
references.pmid: 10706212
references.citation: Curr Opin Neurobiol. 2006 Dec;16(6):710-5
references.pmid: 17011181
references.citation: J Physiol. 1973 Jul;232(2):357-74
references.pmid: 4727085
references.citation: Neural Netw. 2002 Jun-Jul;15(4-6):535-47
references.pmid: 12371510
references.citation: Neuropeptides. 2016 Feb;55:79-89
references.pmid: 26454711
references.citation: Nature. 1986 Feb 27-Mar 5;319(6056):774-6
references.pmid: 2869411
references.citation: J Physiol. 1973 Jul;232(2):331-56
references.pmid: 4727084
references.citation: Curr Opin Neurobiol. 2014 Dec;29:48-56
references.pmid: 24907657
references.citation: Neuroscience. 2002;111(4):815-35
references.pmid: 12031406
references.citation: Neural Netw. 2002 Jun-Jul;15(4-6):495-506
references.pmid: 12371507
references.citation: HFSP J. 2007 May;1(1):30-40
references.pmid: 19404458
references.citation: Neuroscience. 2011 Sep 29;192:91-101
references.pmid: 21736924
references.citation: Proc Natl Acad Sci U S A. 1989 Nov;86(22):9015-9
references.pmid: 2573073
references.citation: Neuroimage. 2015 Feb 1;106:222-37
references.pmid: 25462794
references.citation: Nat Rev Neurosci. 2003 Mar;4(3):193-202
references.pmid: 12612632
references.citation: Biol Cybern. 2002 Dec;87(5-6):404-15
references.pmid: 12461630
references.citation: J Neurosci. 2001 Aug 15;21(16):5984-92
references.pmid: 11487621
references.citation: PLoS Comput Biol. 2008 Oct;4(10):e1000180
references.pmid: 18846203
references.citation: Behav Neurosci. 1992 Jun;106(3):439-46
references.pmid: 1616610
references.citation: Science. 1997 Jan 10;275(5297):213-5
references.pmid: 8985014
references.citation: Annu Rev Psychol. 2006;57:87-115
references.pmid: 16318590
references.citation: Neuron. 2001 Dec 20;32(6):1149-64
references.pmid: 11754844
references.citation: J Neurosci. 2013 Jun 5;33(23):9565-75
references.pmid: 23739954
references.citation: Neuron. 2006 Jul 20;51(2):227-38
references.pmid: 16846857
references.citation: Annu Rev Neurosci. 2008;31:69-89
references.pmid: 18284371
references.citation: Annu Rev Neurosci. 2001;24:139-66
references.pmid: 11283308
references.citation: Nat Commun. 2013;4:2760
references.pmid: 24217681
references.citation: Phys Rev E Stat Nonlin Soft Matter Phys. 2004 Apr;69(4 Pt 1):041909
references.pmid: 15169045
references.citation: J Comput Neurosci. 2000 May-Jun;8(3):183-208
references.pmid: 10809012
references.citation: PLoS Comput Biol. 2009 Dec;5(12):e1000586
references.pmid: 19997492
references.citation: Nat Neurosci. 2003 Jun;6(6):587-92
references.pmid: 12740581
references.citation: PLoS Comput Biol. 2010 Feb;6(2):e1000670
references.pmid: 20169176
references.citation: Exp Brain Res. 1991;85(3):491-500
references.pmid: 1915708
references.citation: J Neurosci. 2008 Mar 5;28(10):2435-46
references.pmid: 18322089
references.citation: J Neurophysiol. 2007 Dec;98(6):3648-65
references.pmid: 17928565
references.citation: J Neurophysiol. 2001 Jan;85(1):117-24
references.pmid: 11152712
references.citation: Exp Brain Res. 1998 Aug;121(3):350-4
references.pmid: 9746140
references.citation: Science. 1996 Dec 6;274(5293):1724-6
references.pmid: 8939866
references.citation: J Comp Physiol Psychol. 1979 Feb;93(1):74-104
references.pmid: 221551
references.citation: Nature. 1968 Aug 17;219(5155):736-7
references.pmid: 5667068
references.citation: J Neurosci. 2010 Oct 6;30(40):13326-37
references.pmid: 20926659
references.citation: Physiol Rev. 1998 Jan;78(1):189-225
references.pmid: 9457173
references.citation: PLoS Comput Biol. 2009 Jan;5(1):e1000259
references.pmid: 19148264
references.citation: Annu Rev Neurosci. 2005;28:357-76
references.pmid: 16022600
references.citation: Nat Neurosci. 2000 Sep;3(9):919-26
references.pmid: 10966623
references.citation: Science. 2009 May 22;324(5930):1080-4
references.pmid: 19389999
references.citation: PLoS One. 2012;7(5):e37372
references.pmid: 22655041
references.citation: Proc Natl Acad Sci U S A. 2005 Jun 28;102(26):9335-40
references.pmid: 15961542
references.citation: J Neurophysiol. 1992 Jan;67(1):145-63
references.pmid: 1552316
references.citation: Neural Comput. 2007 Feb;19(2):371-403
references.pmid: 17206869
references.citation: Neuron. 2007 Sep 20;55(6):919-29
references.pmid: 17880895
references.citation: Nature. 2012 Feb 2;482(7383):85-8
references.pmid: 22258508
references.citation: J Neurosci. 1993 Mar;13(3):900-13
references.pmid: 8441015
references.citation: Neural Comput. 2009 Nov;21(11):2991-3009
references.pmid: 19635018
references.citation: Science. 2008 Aug 8;321(5890):848-51
references.pmid: 18687967
references.citation: Neuron. 2001 Oct 25;32(2):339-50
references.pmid: 11684002
references.citation: Trends Neurosci. 2011 Oct;34(10):536-47
references.pmid: 21851992
references.citation: Neural Comput. 2007 Aug;19(8):2245-79
references.pmid: 17571943
references.citation: IEEE Trans Neural Netw. 2004 May;15(3):639-52
references.pmid: 15384552
references.citation: Mol Brain. 2010;3:15
references.pmid: 20465834
references.citation: Behav Brain Res. 2000 Nov;115(2):205-18
references.pmid: 11000421
references.citation: Neuroscience. 1983 Apr;8(4):791-7
references.pmid: 6306504
references.citation: Trends Neurosci. 1998 May;21(5):181-8
references.pmid: 9610879
references.citation: Nature. 1996 Sep 5;383(6595):76-81
references.pmid: 8779718
references.citation: Philos Trans R Soc Lond B Biol Sci. 2003 Apr 29;358(1432):607-11
references.pmid: 12740102
references.citation: J Neurosci. 2003 May 15;23(10):4173-81
references.pmid: 12764105
references.citation: J Neurosci. 1998 Dec 15;18(24):10464-72
references.pmid: 9852584
references.citation: Prog Neurobiol. 2003 Apr;69(6):375-90
references.pmid: 12880632
references.citation: Nat Neurosci. 2000 Nov;3 Suppl:1178-83
references.pmid: 11127835
references.citation: Trends Cogn Sci. 2010 Oct;14(10):464-72
references.pmid: 20829095
references.citation: Nat Neurosci. 1998 Aug;1(4):304-9
references.pmid: 10195164