title: Neural modularity helps organisms evolve to learn new skills without forgetting old skills.
abstract: A long-standing goal in artificial intelligence is creating agents that can learn a variety of different skills for different problems. In the artificial intelligence subfield of neural networks, a barrier to that goal is that when agents learn a new skill they typically do so by losing previously acquired skills, a problem called catastrophic forgetting. That occurs because, to learn the new task, neural learning algorithms change connections that encode previously acquired skills. How networks are organized critically affects their learning dynamics. In this paper, we test whether catastrophic forgetting can be reduced by evolving modular neural networks. Modularity intuitively should reduce learning interference between tasks by separating functionality into physically distinct modules in which learning can be selectively turned on or off. Modularity can further improve learning by having a reinforcement learning module separate from sensory processing modules, allowing learning to happen only in response to a positive or negative reward. In this paper, learning takes place via neuromodulation, which allows agents to selectively change the rate of learning for each neural connection based on environmental stimuli (e.g. to alter learning in specific locations based on the task at hand). To produce modularity, we evolve neural networks with a cost for neural connections. We show that this connection cost technique causes modularity, confirming a previous result, and that such sparsely connected, modular networks have higher overall performance because they learn new skills faster while retaining old skills more and because they have a separate reinforcement learning module. Our results suggest (1) that encouraging modularity in neural networks may help us overcome the long-standing barrier of networks that cannot learn new skills without forgetting old ones, and (2) that one benefit of the modularity ubiquitous in the brains of natural animals might be to alleviate the problem of catastrophic forgetting.
authors.affiliation: Department of Computer and Information Science, Norwegian University of Science and Technology, Trondheim, Norway.
authors.forename: Kai Olav
authors.initials: KO
authors.lastname: Ellefsen
authors.affiliation: Sorbonne Université UPMC Univ Paris 06, UMR 7222, ISIR, Paris, France; CNRS, UMR 7222, ISIR, Paris, France.
authors.forename: Jean-Baptiste
authors.initials: JB
authors.lastname: Mouret
authors.affiliation: Computer Science Department, University of Wyoming, Laramie, Wyoming, United States of America.
authors.forename: Jeff
authors.initials: J
authors.lastname: Clune
mesh_terms: D000818:Animals; D032701:Biomimetics / Q000379:methods; D003198:Computer Simulation; D006801:Humans; D007266:Inhibition, Psychological; D007858:Learning / Q000502:physiology*; D011939:Mental Recall / Q000502:physiology*; D008959:Models, Neurological; D009415:Nerve Net / Q000502:physiology*; D016571:Neural Networks, Computer; D012153:Retention, Psychology / Q000502:physiology*; D013647:Task Performance and Analysis
references.citation: PLoS Comput Biol. 2008 Feb;4(2):e23
references.pmid: 18266463
references.citation: Nature. 2004 Oct 14;431(7010):796-803
references.pmid: 15483601
references.citation: Proc Biol Sci. 2009 Sep 7;276(1670):3201-8
references.pmid: 19535373
references.citation: Artif Life. 2011 Winter;17(1):33-50
references.pmid: 21087148
references.citation: PLoS Comput Biol. 2010 Mar;6(3):e1000719
references.pmid: 20360969
references.citation: Front Psychol. 2013 Aug 05;4:504
references.pmid: 23935590
references.citation: PLoS One. 2013;8(11):e79138
references.pmid: 24236099
references.citation: Evol Comput. 2002 Summer;10(2):99-127
references.pmid: 12180173
references.citation: J Exp Psychol. 1959 Aug;58:97-105
references.pmid: 13796886
references.citation: Evol Comput. 2001 Winter;9(4):495-524
references.pmid: 11709106
references.citation: Psychol Rev. 1990 Apr;97(2):285-308
references.pmid: 2186426
references.citation: Nature. 2001 Feb 22;409(6823):1102-9
references.pmid: 11234024
references.citation: Proc Biol Sci. 2013 Mar 22;280(1755):20122863
references.pmid: 23363632
references.citation: Ann R Coll Surg Engl. 2004 Sep;86(5):334-8
references.pmid: 15333167
references.citation: Evol Comput. 2011 Spring;19(1):1-23
references.pmid: 20583909
references.citation: Nat Rev Genet. 2008 Jan;9(1):75-82
references.pmid: 18059367
references.citation: Annu Rev Neurosci. 1996;19:109-39
references.pmid: 8833438
references.citation: Trends Cogn Sci. 1999 Apr;3(4):128-135
references.pmid: 10322466
references.citation: Phys Rev Lett. 2009 Apr 17;102(15):152001
references.pmid: 19518620
references.citation: Nat Rev Genet. 2007 Dec;8(12):921-31
references.pmid: 18007649
references.citation: Artif Life. 1999 Summer;5(3):203-23
references.pmid: 10648951
references.citation: Proc Natl Acad Sci U S A. 2013 Dec 24;110(52):E5078-87
references.pmid: 24324147
references.citation: Artif Life. 2003 Spring;9(2):93-130
references.pmid: 12906725
references.citation: Evol Anthropol. 2011 Sep-Oct;20(5):169-80
references.pmid: 22034235
references.citation: Evol Comput. 2012 Spring;20(1):91-133
references.pmid: 21838553
references.citation: Proc Natl Acad Sci U S A. 2004 Jan 27;101(4):1081-6
references.pmid: 14722353
references.citation: Proc Natl Acad Sci U S A. 2006 Jun 6;103(23):8577-82
references.pmid: 16723398
references.citation: Psychol Rev. 1995 Jul;102(3):419-57
references.pmid: 7624455
references.citation: Brain. 1997 Apr;120 ( Pt 4):701-22
references.pmid: 9153131
references.citation: Phys Rev E Stat Nonlin Soft Matter Phys. 2004 Aug;70(2 Pt 2):025101
references.pmid: 15447530
references.citation: Curr Opin Neurobiol. 2001 Dec;11(6):757-64
references.pmid: 11741030
references.citation: Prog Neurobiol. 2003 Apr;69(6):375-90
references.pmid: 12880632
references.citation: Proc Natl Acad Sci U S A. 2005 Sep 27;102(39):13773-8
references.pmid: 16174729
references.citation: Phys Rev Lett. 2008 Mar 21;100(11):118703
references.pmid: 18517839