title: Meta-learning in reinforcement learning.
abstract: Meta-parameters in reinforcement learning should be tuned to the environmental dynamics and the animal performance. Here, we propose a biologically plausible meta-reinforcement learning algorithm for tuning these meta-parameters in a dynamic, adaptive manner. We tested our algorithm in both a simulation of a Markov decision task and in a non-linear control task. Our results show that the algorithm robustly finds appropriate meta-parameter values, and controls the meta-parameter time course, in both static and dynamic environments. We suggest that the phasic and tonic components of dopamine neuron firing can encode the signal required for meta-learning of reinforcement learning.
authors.affiliation: CREST, Japan Science and Technology Corporation, ATR, Human Information Science Laboratories, 2-2-2 Hikaridai, Seika-cho, Soraku-gun, 619-0288, Kyoto, Japan. nicolas@atr.co.jp
authors.forename: Nicolas
authors.initials: N
authors.lastname: Schweighofer
authors.forename: Kenji
authors.initials: K
authors.lastname: Doya
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D003662:Decision Theory; D008390:Markov Chains; D012054:Reinforcement, Psychology; D013269:Stochastic Processes