title: An information-theoretic analysis of return maximization in reinforcement learning.
abstract: We present a general analysis of return maximization in reinforcement learning. This analysis does not require assumptions of Markovianity, stationarity, and ergodicity for the stochastic sequential decision processes of reinforcement learning. Instead, our analysis assumes the asymptotic equipartition property fundamental to information theory, providing a substantially different view from that in the literature. As our main results, we show that return maximization is achieved by the overlap of typical and best sequence sets, and we present a class of stochastic sequential decision processes with the necessary condition for return maximization. We also describe several examples of best sequences in terms of return maximization in the class of stochastic sequential decision processes, which satisfy the necessary condition.
authors.affiliation: Graduate School of Information Sciences, Hiroshima City University, Hiroshima, Japan. kiwata@hiroshima-cu.ac.jp
authors.forename: Kazunori
authors.initials: K
authors.lastname: Iwata
mesh_terms: D001185:Artificial Intelligence; D003661:Decision Support Techniques; D007257:Information Theory; D008390:Markov Chains; D016571:Neural Networks, Computer; D011337:Probability Learning; D012054:Reinforcement, Psychology; D013269:Stochastic Processes