title: Reward-based training of recurrent neural networks for cognitive and value-based tasks.
abstract: Trained neural network models, which exhibit features of neural activity recorded from behaving animals, may provide insights into the circuit mechanisms of cognitive functions through systematic analysis of network activity and connectivity. However, in contrast to the graded error signals commonly used to train networks through supervised learning, animals learn from reward feedback on definite actions through reinforcement learning. Reward maximization is particularly relevant when optimal behavior depends on an animal's internal judgment of confidence or subjective preferences. Here, we implement reward-based training of recurrent neural networks in which a value network guides learning by using the activity of the decision network to predict future reward. We show that such models capture behavioral and electrophysiological findings from well-known experimental paradigms. Our work provides a unified framework for investigating diverse cognitive and value-based computations, and predicts a role for value representation that is essential for learning, but not executing, a task.
authors.affiliation: Center for Neural Science, New York University, New York, United States.
authors.forename: H Francis
authors.initials: HF
authors.lastname: Song
authors.affiliation: Center for Neural Science, New York University, New York, United States.
authors.forename: Guangyu R
authors.initials: GR
authors.lastname: Yang
authors.affiliation: Center for Neural Science, New York University, New York, United States.|NYU-ECNU Institute of Brain and Cognitive Science, NYU Shanghai, Shanghai, China.
authors.forename: Xiao-Jing
authors.identifier: 0000-0003-3124-8474
authors.initials: XJ
authors.lastname: Wang
mesh_terms: D000818:Animals; D001522:Behavior, Animal; D002755:Choice Behavior; D003071:Cognition; D003213:Conditioning, Psychological; D003657:Decision Making; D007858:Learning; D008959:Models, Neurological; D012201:Reward
keywords: decision making; neuroscience; orbitofrontal cortex; prefrontal cortex; recurrent neural network; reinforcement learning; working memory