Meta-learning in reinforcement learning.
Meta-parameters in reinforcement learning should be tuned to the environmental dynamics and the animal performance. Here, we propose a biologically plausible meta-reinforcement learning algorithm for tuning these meta-parameters in a dynamic, adaptive manner. We tested our algorithm in both a simulation of a Markov decision task and in a non-linear control task. Our results show that the algorithm robustly finds appropriate meta-parameter values, and controls the meta-parameter time course, in both static and dynamic environments. We suggest that the phasic and tonic components of dopamine neuron firing can encode the signal required for meta-learning of reinforcement learning.
Schweighofer
Nicolas
N
CREST, Japan Science and Technology Corporation, ATR, Human Information Science Laboratories, 2-2-2 Hikaridai, Seika-cho, Soraku-gun, 619-0288, Kyoto, Japan. nicolas@atr.co.jp
Doya
Kenji
K
D000465:Algorithms; D001185:Artificial Intelligence; D003662:Decision Theory; D008390:Markov Chains; D012054:Reinforcement, Psychology; D013269:Stochastic Processes