title: Acceleration of reinforcement learning by policy evaluation using nonstationary iterative method.
abstract: Typical methods for solving reinforcement learning problems iterate two steps, policy evaluation and policy improvement. This paper proposes algorithms for the policy evaluation to improve learning efficiency. The proposed algorithms are based on the Krylov Subspace Method (KSM), which is a nonstationary iterative method. The algorithms based on KSM are tens to hundreds times more efficient than existing algorithms based on the stationary iterative methods. Algorithms based on KSM are far more efficient than they have been generally expected. This paper clarifies what makes algorithms based on KSM makes more efficient with numerical examples and theoretical discussions.
authors.forename: Kei
authors.initials: K
authors.lastname: Senda
authors.forename: Suguru
authors.initials: S
authors.lastname: Hattori
authors.forename: Toru
authors.initials: T
authors.lastname: Hishinuma
authors.forename: Takehisa
authors.initials: T
authors.lastname: Kohda
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D003198:Computer Simulation; D003661:Decision Support Techniques; D005246:Feedback; D008962:Models, Theoretical; D011340:Problem Solving; D012054:Reinforcement, Psychology