title: Divide et impera: subgoaling reduces the complexity of probabilistic inference and problem solving.
abstract: It has long been recognized that humans (and possibly other animals) usually break problems down into smaller and more manageable problems using subgoals. Despite a general consensus that subgoaling helps problem solving, it is still unclear what the mechanisms guiding online subgoal selection are during the solution of novel problems for which predefined solutions are not available. Under which conditions does subgoaling lead to optimal behaviour? When is subgoaling better than solving a problem from start to finish? Which is the best number and sequence of subgoals to solve a given problem? How are these subgoals selected during online inference? Here, we present a computational account of subgoaling in problem solving. Following Occam's razor, we propose that good subgoals are those that permit planning solutions and controlling behaviour using less information resources, thus yielding parsimony in inference and control. We implement this principle using approximate probabilistic inference: subgoals are selected using a sampling method that considers the descriptive complexity of the resulting sub-problems. We validate the proposed method using a standard reinforcement learning benchmark (four-rooms scenario) and show that the proposed method requires less inferential steps and permits selecting more compact control programs compared to an equivalent procedure without subgoaling. Furthermore, we show that the proposed method offers a mechanistic explanation of the neuronal dynamics found in the prefrontal cortex of monkeys that solve planning problems. Our computational framework provides a novel integrative perspective on subgoaling and its adaptive advantages for planning, control and learning, such as for example lowering cognitive effort and working memory load.
authors: Institute for High Performance Computing and Networking, National Research Council, Via Pietro Castellino, 111, 80131 Naples, Italy.
authors: Domenico
authors: D
authors: Maisto
authors: Institute of Cognitive Sciences and Technologies, National Research Council, Via S. Martino della Battaglia, 44, 00185 Rome, Italy.
authors: Francesco
authors: F
authors: Donnarumma
authors: Institute of Cognitive Sciences and Technologies, National Research Council, Via S. Martino della Battaglia, 44, 00185 Rome, Italy giovanni.pezzulo@istc.cnr.it.
authors: Giovanni
authors: http://orcid.org/0000-0001-6813-8282
authors: G
authors: Pezzulo
mesh_terms: D000465:Algorithms; D000818:Animals; D001499:Bayes Theorem; D001921:Brain / Q000502:physiology; D003198:Computer Simulation; D000882:Haplorhini; D006801:Humans; D020407:Internet; D007858:Learning; D008390:Markov Chains; D009010:Monte Carlo Method; D009474:Neurons / Q000378:metabolism*; D009488:Neurosciences / Q000379:methods; D017397:Prefrontal Cortex / Q000502:physiology; D011336:Probability; D011340:Problem Solving
keywords: active inference; hierarchies; model-based reinforcement learning; planning-as-inference; problem solving; subgoals
references: PLoS Comput Biol. 2008 Nov;4(11):e1000209
references: 19008936
references: Front Psychol. 2013 Mar 04;4:92
references: 23459512
references: Neural Netw. 1997 Jul;10(5):857-873
references: 12662875
references: Nat Rev Neurosci. 2010 Feb;11(2):127-38
references: 20068583
references: PLoS One. 2009 Jul 29;4(7):e6421
references: 19641614
references: Philos Trans R Soc Lond B Biol Sci. 1982 Jun 25;298(1089):199-209
references: 6125971
references: Neuron. 2006 May 18;50(4):631-41
references: 16701212
references: Trends Cogn Sci. 2012 Oct;16(10):485-8
references: 22940577
references: Biol Cybern. 2012 Oct;106(8-9):523-41
references: 22864468
references: Trends Cogn Sci. 2008 May;12(5):201-8
references: 18420448
references: Cereb Cortex. 2005 Oct;15(10):1535-46
references: 15703260
references: PLoS Comput Biol. 2014 Aug 14;10(8):e1003779
references: 25122479
references: Psychol Rev. 2012 Jan;119(1):120-54
references: 22229491
references: Wiley Interdiscip Rev Cogn Sci. 2013 Jul;4(4):341-356
references: 26304223
references: Adv Neural Inf Process Syst. 2009;21:169-176
references: 25258502
references: Annu Rev Neurosci. 2001;24:167-202
references: 11283309
references: Philos Trans R Soc Lond B Biol Sci. 2014 Nov 5;369(1655):
references: 25267825
references: Science. 2012 Jan 6;335(6064):108-11
references: 22157081
references: Nat Neurosci. 2005 Dec;8(12):1704-11
references: 16286932
references: Cognition. 2009 Dec;113(3):262-280
references: 18926527
references: J Neurosci. 2011 Apr 6;31(14):5526-39
references: 21471389
references: Front Neurosci. 2011 Jun 30;5:79
references: 21747755
references: Psychol Res. 2009 Jul;73(4):559-77
references: 19347359
references: Bull Math Biol. 2013 Jul;75(7):1181-206
references: 23666484
references: Neuron. 2012 May 24;74(4):656-62
references: 22632724
references: J Neurosci. 2007 Aug 1;27(31):8170-3
references: 17670961
references: Philos Trans R Soc Lond B Biol Sci. 2014 Nov 5;369(1655):
references: 25267813
references: Trends Cogn Sci. 2014 Dec;18(12):647-57
references: 25156191