A new criterion using information gain for action selection strategy in reinforcement learning.
In this paper, we regard the sequence of returns as outputs from a parametric compound source. Utilizing the fact that the coding rate of the source shows the amount of information about the return, we describe l-learning algorithms based on the predictive coding idea for estimating an expected information gain concerning future information and give a convergence proof of the information gain. Using the information gain, we propose the ratio w of return loss to information gain as a new criterion to be used in probabilistic action-selection strategies. In experimental results, we found that our w-based strategy performs well compared with the conventional Q-based strategy.
Iwata
Kazunori
K
Graduate School of Informatics, Kyoto University, Kyoto 606-8501, Japan. kiwata@sys.i.kyoto-u.ac.jp
Ikeda
Kazushi
K
Sakai
Hideaki
H
D000465:Algorithms; D001185:Artificial Intelligence; D003198:Computer Simulation; D003661:Decision Support Techniques; D016247:Information Storage and Retrieval / Q000379:methods*; D007257:Information Theory; D015233:Models, Statistical; D016571:Neural Networks, Computer; D011337:Probability Learning; D012054:Reinforcement, Psychology