title: Internally generated sequences in learning and executing goal-directed behavior.
abstract: A network of brain structures including hippocampus (HC), prefrontal cortex, and striatum controls goal-directed behavior and decision making. However, the neural mechanisms underlying these functions are unknown. Here, we review the role of 'internally generated sequences': structured, multi-neuron firing patterns in the network that are not confined to signaling the current state or location of an agent, but are generated on the basis of internal brain dynamics. Neurophysiological studies suggest that such sequences fulfill functions in memory consolidation, augmentation of representations, internal simulation, and recombination of acquired information. Using computational modeling, we propose that internally generated sequences may be productively considered a component of goal-directed decision systems, implementing a sampling-based inference engine that optimizes goal acquisition at multiple timescales of on-line choice, action control, and learning.
authors.affiliation: Institute of Cognitive Sciences and Technologies, National Research Council, Via San Martino della Battaglia 44, 00185 Roma, Italy.
authors.forename: Giovanni
authors.initials: G
authors.lastname: Pezzulo
authors.affiliation: Department of Biology and Centre for Theoretical Neuroscience, University of Waterloo, 200 University Avenue West, Waterloo, ON N2L 3G1, Canada.
authors.forename: Matthijs A A
authors.initials: MA
authors.lastname: van der Meer
authors.affiliation: Swammerdam Institute for Life Sciences - Center for Neuroscience, University of Amsterdam, Science Park 904, 1098 XH Amsterdam, The Netherlands; Amsterdam Brain and Cognition, Research Priority Program Brain and Cognition, Nieuwe Achtergracht 129, 1018 WS Amsterdam, The Netherlands.
authors.forename: Carien S
authors.initials: CS
authors.lastname: Lansink
authors.affiliation: Swammerdam Institute for Life Sciences - Center for Neuroscience, University of Amsterdam, Science Park 904, 1098 XH Amsterdam, The Netherlands; Amsterdam Brain and Cognition, Research Priority Program Brain and Cognition, Nieuwe Achtergracht 129, 1018 WS Amsterdam, The Netherlands. Electronic address: c.m.a.pennartz@uva.nl.
authors.forename: Cyriel M A
authors.initials: CM
authors.lastname: Pennartz
mesh_terms: D000818:Animals; D001921:Brain / Q000502:physiology*; D003657:Decision Making / Q000502:physiology*; D006040:Goals; D006184:Habits; D006801:Humans; D007858:Learning / Q000502:physiology*; D008568:Memory / Q000502:physiology; D011597:Psychomotor Performance / Q000502:physiology*
keywords: decision making; forward sweep; generative models; hippocampus; inference; prospection; reinforcement learning; replay; spatial navigation; theta rhythm; ventral striatum