Toward an Integration of Deep Learning and Neuroscience.
Neuroscience has focused on the detailed implementation of computation, studying neural codes, dynamics and circuits. In machine learning, however, artificial neural networks tend to eschew precisely designed codes, dynamics or circuits in favor of brute force optimization of a cost function, often using simple and relatively uniform initial architectures. Two recent developments have emerged within machine learning that create an opportunity to connect these seemingly divergent perspectives. First, structured architectures are used, including dedicated systems for attention, recursion and various forms of short- and long-term memory storage. Second, cost functions and training procedures have become more complex and are varied across layers and over time. Here we think about the brain in terms of these ideas. We hypothesize that (1) the brain optimizes cost functions, (2) the cost functions are diverse and differ across brain locations and over development, and (3) optimization operates within a pre-structured architecture matched to the computational problems posed by behavior. In support of these hypotheses, we argue that a range of implementations of credit assignment through multiple layers of neurons are compatible with our current knowledge of neural circuitry, and that the brain's specialized systems can be interpreted as enabling efficient optimization for specific problem classes. Such a heterogeneously optimized system, enabled by a series of interacting cost functions, serves to make learning data-efficient and precisely targeted to the needs of the organism. We suggest directions by which neuroscience could seek to refine and test these hypotheses.
Synthetic Neurobiology Group, Massachusetts Institute of Technology, Media Lab Cambridge, MA, USA.
Adam H
AH
Marblestone
Google Deepmind London, UK.
Greg
G
Wayne
Rehabilitation Institute of Chicago, Northwestern University Chicago, IL, USA.
Konrad P
KP
Kording
cognitive architecture; cost functions; neural networks; neuroscience
Proc Natl Acad Sci U S A. 2014 Oct 7;111(40):14332-41
25205811
Neural Comput. 1991 Summer;3(2):194-200
31167302
Comput Intell Neurosci. 2013;2013:149329
23935605
Science. 2015 Nov 27;350(6264):aac9462
26612957
Psychol Sci. 2010 Oct;21(10):1518-24
20855906
J Neurosci. 2010 Oct 13;30(41):13850-60
20943925
Curr Opin Neurobiol. 2014 Apr;25:156-63
24509098
Prog Brain Res. 2015;219:1-19
26072231
Trends Neurosci. 2013 Mar;36(3):141-51
23273272
Neural Comput. 2005 Oct;17(10):2176-214
16105222
Dev Sci. 2016 May;19(3):427-39
26190343
Neural Comput. 2006 Jul;18(7):1527-54
16764513
J Neurosci Res. 2010 Nov 1;88(14):2991-3001
20544831
J Neurosci. 2010 Nov 24;30(47):15801-10
21106819
Front Syst Neurosci. 2008 Nov 24;2:4
19104670
Front Neural Circuits. 2013 Jun 19;7:106
23801941
PLoS One. 2013 Nov 13;8(11):e79659
24236151
Neural Comput. 2001 Dec;13(12):2709-41
11705408
PLoS Comput Biol. 2007 Aug;3(8):e166
17784780
Nat Neurosci. 2016 Mar;19(3):356-65
26906502
Cold Spring Harb Perspect Biol. 2012 Jan 01;4(1):a005736
22086977
Nat Neurosci. 2015 Jan;18(1):138-44
25420067
Neuron. 2015 Jul 1;87(1):231-43
26139376
Nat Neurosci. 2002 Nov;5(11):1226-35
12404008
Nat Commun. 2016 May 13;7:11552
27174042
Front Neuroanat. 2011 Dec 20;5:65
22194717
Neural Netw. 2000 Jan;13(1):1-9
10935452
Nat Neurosci. 2016 Mar;19(3):350-5
26906501
Trends Cogn Sci. 2004 Feb;8(2):49-51
15588805
Science. 2015 Mar 6;347(6226):1138-42
25700174
Science. 2014 Apr 25;344(6182):424-7
24763592
Neuron. 2013 Jun 19;78(6):1090-101
23707613
J Neurosci. 1993 Jan;13(1):334-50
8423479
Trends Cogn Sci. 2007 Jul;11(7):299-306
17548229
Nat Neurosci. 2014 Mar;17(3):383-90
24487231
Science. 1989 Aug 11;245(4918):605-15
2762813
Science. 2006 Oct 6;314(5796):91-4
17023651
PLoS Comput Biol. 2016 Jun 10;12(6):e1004967
27286251
Neural Comput. 2013 Nov;25(11):2815-32
24001342
Front Comput Neurosci. 2014 Jan 09;7:194
24409138
Curr Opin Neurobiol. 2016 Apr;37:114-120
26921828
Proc Natl Acad Sci U S A. 2014 Apr 22;111(16):6063-8
24711417
Prog Brain Res. 2005;149:107-26
16226580
Elife. 2015 Jun 26;4:e06063
26112828
Nature. 2005 Jul 7;436(7047):71-7
16001064
Proc Natl Acad Sci U S A. 1984 May;81(10):3088-92
6587342
Curr Opin Neurobiol. 2015 Aug;33:166-73
25978563
Nat Rev Neurosci. 2006 Jul;7(7):563-74
16791145
Cereb Cortex. 1991 Jan-Feb;1(1):1-47
1822724
Neuron. 2007 Dec 6;56(5):866-79
18054862
PLoS Comput Biol. 2014 Nov 06;10(11):e1003915
25375136
Neuron. 2016 May 18;90(4):893-903
27196978
Annu Rev Psychol. 2016;67:159-86
26393869
Neuron. 2003 Dec 18;40(6):1063-73
14687542
Neural Comput. 2008 Jun;20(6):1427-51
18254695
J Comp Neurol. 2011 Dec 15;519(18):3672-83
21618232
PLoS Comput Biol. 2009 Oct;5(10):e1000532
19816557
Nature. 2015 May 28;521(7553):436-44
26017442
Curr Opin Neurobiol. 2006 Dec;16(6):710-5
17011181
Nature. 2005 Feb 24;433(7028):873-6
15729344
PLoS Comput Biol. 2015 Mar 05;11(3):e1004060
25742003
Elife. 2015 Dec 18;4:
26682652
Nature. 2014 Aug 28;512(7515):423-6
25164754
Prog Neurobiol. 2013 Apr;103:115-30
22307059
Proc Natl Acad Sci U S A. 2012 Oct 30;109(44):18215-20
23012418
Neural Netw. 2016 May;77:95-106
26963256
Neuron. 2016 Jun 1;90(5):1028-42
27210551
Nature. 2016 May 23;534(7605):115-8
27251287
Nat Methods. 2013 Jun;10(6):483-90
23866325
Psychol Rev. 1956 Mar;63(2):81-97
13310704
IEEE Trans Neural Netw. 1995;6(3):623-41
18263348
Cereb Cortex. 2003 Nov;13(11):1185-95
14576210
Proc Natl Acad Sci U S A. 1987 Sep;84(17):6297-301
3114747
Behav Brain Sci. 2017 Jan;40:e253
27881212
Proc Natl Acad Sci U S A. 2009 Jul 14;106(28):11478-83
19574462
Trends Cogn Sci. 2010 May;14(5):208-15
20207189
Proc Natl Acad Sci U S A. 2013 Oct 8;110(41):16390-5
24062434
PLoS Comput Biol. 2014 Jun 12;10(6):e1003577
24921249
Psychol Rev. 2007 Jul;114(3):632-56
17638499
Nature. 1989 Jan 12;337(6203):129-32
2911347
Elife. 2014 Dec 23;3:e04577
25535793
Neuron. 2009 Aug 27;63(4):544-57
19709635
Elife. 2014 Jun 16;3:
24935934
J Cogn Neurosci. 2004 Jun;16(5):856-77
15200713
Cereb Cortex. 2007 Sep;17 Suppl 1:i41-50
17726003
J Neurosci. 2015 Jul 8;35(27):10005-14
26157000
Biol Cybern. 2001 Jun;84(6):401-10
11417052
Ann N Y Acad Sci. 1999 Jun 30;882:119-27; discussion 128-34
10415890
Proc Natl Acad Sci U S A. 2015 Sep 1;112(35):11084-9
26283366
Neuron. 2006 Jul 20;51(2):227-38
16846857
Trends Cogn Sci. 2010 Feb;14(2):64-71
20060771
PLoS Comput Biol. 2016 Jun 16;12(6):e1004895
27309381
Front Comput Neurosci. 2012 Sep 28;6:73
23060784
Nature. 1998 Aug 20;394(6695):780-4
9723616
PLoS Comput Biol. 2013 Apr;9(4):e1003035
23637588
Trends Cogn Sci. 2016 Aug;20(8):589-604
27388875
Neuron. 2010 Nov 4;68(3):362-85
21040841
PLoS Comput Biol. 2015 Oct 23;11(10):e1004489
26496502
Front Neuroanat. 2011 May 23;5:30
21647397
Neural Netw. 2013 Sep;45:94-100
23541926
Nat Rev Neurosci. 2010 Feb;11(2):127-38
20068583
Annu Rev Neurosci. 2004;27:419-51
15217339
Curr Opin Neurobiol. 2015 Apr;31:119-26
25291080
Trends Neurosci. 2004 Dec;27(12):712-9
15541511
Cell. 2016 Mar 24;165(1):192-206
27015312
J Exp Psychol Hum Percept Perform. 2011 Aug;37(4):1032-50
21517211
PLoS Comput Biol. 2007 Jan 19;3(1):e165
17238280
Nature. 1989 Feb 9;337(6207):549-55
2915704
Cell. 2015 Aug 27;162(5):1155-68
26317475
J Neurosci. 2009 Oct 21;29(42):13172-81
19846705
Nature. 2004 Oct 14;431(7010):768-74
15483597
Cognition. 2014 Feb;130(2):204-16
24321623
Science. 1995 May 26;268(5214):1158-61
7761831
Front Synaptic Neurosci. 2010 Jul 21;2:25
21423511
Front Behav Neurosci. 2012 Jul 25;6:38
22848193
J Neurosci. 2006 Oct 11;26(41):10420-9
17035526
Neuron. 2014 Jun 18;82(6):1394-406
24945778
Bull Math Biol. 1990;52(1-2):99-115; discussion 73-97
2185863
Nature. 1997 Nov 20;390(6657):279-81
9384378
Hippocampus. 2015 Jun;25(6):744-52
25786389
Trends Neurosci. 2005 Jul;28(7):364-70
15935487
J Neurosci. 2006 Sep 6;26(36):9117-29
16957068
Science. 2016 Jun 17;352(6292):1464-1468
27313047
Neuroimage. 2008 Jan 15;39(2):647-60
17977024
Science. 2014 Oct 31;346(6209):551-2
25359953
Nature. 1992 Jan 9;355(6356):161-3
1729650
Genetics. 2014 Jan;196(1):17-29
24395823
Neural Comput. 2002 Nov;14(11):2531-60
12433288
Dev Psychol. 2012 Sep;48(5):1215-28
22369335
Neural Comput. 1997 Nov 15;9(8):1735-80
9377276
Nat Neurosci. 2014 Aug;17(8):1031-9
25065440
J Neurosci. 2015 Oct 14;35(41):13912-6
26468192
Science. 1969 Feb 28;163(3870):955-8
4974291
Synthese. 2007 Dec 1;159(3):417-458
19325932
Neural Comput. 2008 Jun;20(6):1473-94
18254696
Neuron. 2016 Apr 6;90(1):128-42
26971945
J Physiol. 2007 Mar 15;579(Pt 3):571-9
17234689
Cell. 2015 Dec 17;163(7):1742-55
26687359
Philos Trans R Soc Lond B Biol Sci. 2014 Nov 5;369(1655):
25267822
J Neurosci. 2006 Sep 20;26(38):9673-82
16988038
Neural Comput. 2003 Feb;15(2):441-54
12590814
Neuron. 2013 Feb 20;77(4):723-35
23439124
PLoS Comput Biol. 2016 Sep 30;12(9):e1005070
27690349
Front Syst Neurosci. 2013 Oct 30;7:74
24198767
Trends Cogn Sci. 2003 Jul;7(7):293-299
12860187
Proc Natl Acad Sci U S A. 2015 Mar 10;112(10):3098-103
25675480
PLoS Comput Biol. 2015 Oct 23;11(10):e1004390
26496457
Nature. 2016 May 02;533(7603):402-6
27135927
Nat Neurosci. 2013 Feb;16(2):130-8
23354386
Front Neural Circuits. 2016 Mar 30;10:23
27065813
Neuron. 2012 Feb 9;73(3):415-34
22325196
Neural Comput. 2002 Apr;14(4):715-70
11936959
Neural Netw. 2004 Jun-Jul;17(5-6):625-32
15288888
J Neurosci. 1997 Jun 1;17(11):4302-11
9151747
Proc Natl Acad Sci U S A. 2007 Apr 10;104(15):6424-9
17404214
Nature. 1998 Sep 17;395(6699):272-4
9751053
Wiley Interdiscip Rev Cogn Sci. 2011 Sep;2(5):580-593
26302308
Cell. 2015 Oct 08;163(2):456-92
26451489
Science. 2007 Mar 23;315(5819):1719-22
17379810
J Neurophysiol. 2007 Oct;98(4):2038-57
17652414
Nature. 2015 Apr 9;520(7546):180-5
25822789
Neural Comput. 2008 Oct;20(10):2526-63
18439138
J Neurosci. 2000 Nov 15;20(22):8238-46
11069929
Neurobiol Learn Mem. 1998 Jul-Sep;70(1-2):119-36
9753592
J Neurosci. 2012 May 9;32(19):6611-20
22573683
PLoS Comput Biol. 2015 Mar 13;11(3):e1004090
25768881
Nat Commun. 2017 Jul 26;8(1):138
28743932
Neuropsychologia. 2010 Mar;48(4):1022-31
19962999
Proc Natl Acad Sci U S A. 2010 Sep 28;107(39):17041-6
20837526
Cell. 1996 Dec 13;87(6):1025-35
8978607
Phys Rev Lett. 1987 Nov 9;59(19):2229-2232
10035458
Proc Natl Acad Sci U S A. 2015 Sep 15;112(37):11732-7
26305927
Neural Comput. 2006 Feb;18(2):245-82
16378515
Behav Brain Res. 1997 Dec;89(1-2):1-34
9475612
Neuron. 2006 May 18;50(4):631-41
16701212
Neuroscience. 2015 Jun 18;296:66-74
25575943
PLoS Comput Biol. 2011 Nov;7(11):e1002211
22096452
Proc Natl Acad Sci U S A. 2013 May 28;110(22):9136-41
23676276
J Neurosci. 1993 Nov;13(11):4700-19
8229193
Neural Netw. 2013 Jan;37:1-47
23149242
Nature. 2001 Mar 29;410(6828):588-92
11279497
Proc Natl Acad Sci U S A. 2005 Jan 18;102(3):880-5
15630093
J Neurophysiol. 2014 Oct 15;112(8):1849-56
25080566
J Physiol. 1969 Jun;202(2):437-70
5784296
Elife. 2015 Aug 06;4:
26245232
Philos Trans R Soc Lond B Biol Sci. 2009 May 12;364(1521):1263-71
19528007
Neuron. 2014 Feb 5;81(3):521-8
24507189
J Neurosci. 2011 Dec 7;31(49):18155-65
22159127
Bioessays. 2012 Jun;34(6):458-65
22396302
PLoS Comput Biol. 2014 Mar 27;10(3):e1003511
24675787
Cereb Cortex. 1996 May-Jun;6(3):406-16
8670667
Neural Comput. 2012 Jun;24(6):1553-68
22364500
Nature. 1987 Jul 2-8;328(6125):73-7
3600775
Trends Cogn Sci. 2016 Apr;20(4):291-303
26944218
Nature. 2013 May 30;497(7451):585-90
23685452
Science. 2012 Nov 30;338(6111):1202-5
23197532
Neuron. 2014 Sep 17;83(6):1239-41
25233303
Trends Cogn Sci. 2016 Jul;20(7):512-534
27315762
Curr Opin Neurobiol. 2016 Apr;37:99-105
26874471
Philos Trans R Soc Lond B Biol Sci. 2005 Jun 29;360(1458):1271-9
16147522
Proc Natl Acad Sci U S A. 2014 Jan 7;111(1):480-5
24367075
Philos Trans R Soc Lond B Biol Sci. 2014 Nov 5;369(1655):
25267814
Science. 1997 Jan 10;275(5297):213-5
8985014
Proc Natl Acad Sci U S A. 2006 Feb 28;103(9):3387-92
16488974
Curr Opin Neurobiol. 2016 Apr;37:7-11
26719992
Proc Natl Acad Sci U S A. 1982 Apr;79(8):2554-8
6953413
Hippocampus. 2000;10(1):1-16
10706212
Cereb Cortex. 2012 Mar;22(3):509-26
21693490
Neuron. 2010 Feb 25;65(4):563-76
20188660
Phys Rev Lett. 2006 Jul 28;97(4):048104
16907616
Proc Natl Acad Sci U S A. 2010 Jan 26;107(4):1648-53
20080534
PLoS Comput Biol. 2013 Apr;9(4):e1003037
23633941
Neural Comput. 2004 Jan;16(1):1-38
15006021
Nat Rev Neurosci. 2005 Jun;6(6):477-83
15928717
Nature. 2007 Nov 22;450(7169):557-9
18033298
Nat Neurosci. 2015 Jul;18(7):1025-33
26075643
Cognition. 2002 Aug;85(1):53-78
12086713
Cereb Cortex. 2014 Mar;24(3):677-90
23146969
Atten Percept Psychophys. 2012 May;74(4):691-702
22271165
Neuron. 2006 Jun 15;50(6):963-74
16772176
Front Comput Neurosci. 2015 Apr 21;9:46
25954191
Front Comput Neurosci. 2013 Jul 18;7:98
23882213
PLoS Biol. 2006 May;4(5):e120
16605306
Curr Opin Neurobiol. 2015 Dec;35:110-8
26298193
Nature. 1996 Jun 13;381(6583):607-9
8637596
Proc Natl Acad Sci U S A. 2009 Dec 15;106(50):21341-6
19926847
Neuron. 1999 Sep;24(1):19-29, 111-25
10677024
Neurobiol Learn Mem. 2004 Nov;82(3):171-7
15464402
Nature. 2015 Sep 17;525(7569):333-8
26352471
Cognition. 2009 Dec;113(3):262-280
18926527
Neuroscience. 1985 Apr;14(4):1039-52
2860616
Neuron. 2009 Sep 24;63(6):889-901
19778516
J Opt Soc Am A Opt Image Sci Vis. 2003 Jul;20(7):1434-48
12868647
Curr Opin Neurobiol. 2007 Aug;17(4):417-22
17707635
Proc Natl Acad Sci U S A. 2009 Dec 1;106(48):20538-43
19918074
Nature. 2013 May 2;497(7447):113-7
23615618
Nat Commun. 2016 Nov 08;7:13276
27824044
Trends Neurosci. 2008 Mar;31(3):130-6
18255165
Neural Comput. 2000 Feb;12(2):337-65
10636946
Nature. 2007 Sep 13;449(7159):223-7
17851526
Nat Neurosci. 2007 Jan;10(1):100-7
17173043
Neural Comput. 2016 Nov;28(11):2474-2504
27626963
Front Psychol. 2013 Feb 20;4:71
23431042
Science. 2007 Oct 26;318(5850):606-10
17962554
Neural Netw. 1998 Oct;11(7-8):1253-1258
12662748
Cognition. 2016 Jul;152:101-107
27043744
Science. 2004 Apr 2;304(5667):78-80
15064413
Front Hum Neurosci. 2010 Nov 02;4:194
21119775
Vision Res. 2010 Oct 28;50(22):2233-47
20493206
Neural Comput. 2003 Oct;15(10):2255-79
14511521
Nat Neurosci. 2006 Nov;9(11):1432-8
17057707
Proc IEEE Inst Electr Electron Eng. 2014 May;102(5):
25328167
Biol Cybern. 1980;36(4):193-202
7370364
PLoS Comput Biol. 2017 Jan 12;13(1):e1005268
28081141
PLoS Comput Biol. 2016 Jun 24;12(6):e1005003
27341100
J Neurosci. 2011 Jul 27;31(30):10787-802
21795531
Front Evol Neurosci. 2012 Apr 13;4:5
22518103
Neuron. 2015 Oct 7;88(1):2-19
26447569
Nature. 2015 Feb 26;518(7540):529-33
25719670
Neural Comput. 2005 Dec;17(12):2699-718
16212768
Trends Neurosci. 2009 Aug;32(8):421-31
19615761
Neural Comput. 2006 Feb;18(2):283-328
16378516
Front Psychol. 2011 Nov 21;2:342
22125544
J Neural Eng. 2011 Dec;8(6):065009
22056418
Science. 1995 Feb 17;267(5200):1028-30
7863330
J Neurophysiol. 2004 Jan;91(1):206-12
12904330
Neural Comput. 2014 Oct;26(10):2163-93
25058706
Psychol Rev. 2010 Apr;117(2):541-74
20438237
Science. 2015 Dec 11;350(6266):1332-8
26659050
Neuron. 2016 Mar 2;89(5):1016-30
26898780
Curr Biol. 2013 May 20;23(10):R448-51
23701688
Neurobiol Learn Mem. 2015 Jan;117:22-33
24752152
Neuron. 2011 Aug 25;71(4):737-49
21867888
Neuron. 2012 Oct 4;76(1):240-56
23040818
PLoS Comput Biol. 2015 Nov 06;11(11):e1004485
26545099
Elife. 2014 Dec 23;3:e04580
25535794
J Neurosci. 2006 Apr 5;26(14):3667-78
16597721
J Neurosci. 2002 Oct 1;22(19):8633-46
12351737
Neural Comput. 2009 Feb;21(2):510-32
19431268
Biol Cybern. 2011 May;104(4-5):251-62
21573688
J Comput Neurosci. 2001 Nov-Dec;11(3):207-15
11796938
Proc Natl Acad Sci U S A. 2013 Nov 5;110(45):18327-32
24145417
PLoS Comput Biol. 2016 Feb 03;12(2):e1004638
26841235
Science. 2013 Apr 12;340(6129):157-61
23580521
Vision Res. 1997 Dec;37(23):3311-25
9425546
Philos Trans R Soc Lond B Biol Sci. 2014 Nov 5;369(1655):
25267813
Annu Rev Neurosci. 2000;23:441-71
10845071
Cereb Cortex. 2007 Oct;17(10):2443-52
17220510
Nature. 1988 Feb 25;331(6158):679-84
3344044
Neural Netw. 1999 Oct;12(7-8):961-974
12662639
Neuron. 2015 Jan 21;85(2):390-401
25556836
Science. 2016 Jun 10;352(6291):1323-6
27284194
Front Comput Neurosci. 2012 Jun 25;6:37
22754523
Curr Biol. 2014 Sep 22;24(18):R921-R929
25247371
Curr Opin Neurobiol. 2010 Dec;20(6):704-16
20850966
Neural Netw. 2015 Jan;61:85-117
25462637
Neural Comput. 2002 Jun;14(6):1233-60
12020444
J Neurosci. 2014 Feb 19;34(8):2871-8
24553928
J Neurosci. 2011 Nov 9;31(45):16353-68
22072687
Proc Natl Acad Sci U S A. 2014 Oct 14;111(41):14930-4
25267641