title: Autonomous reinforcement learning with experience replay.
abstract: This paper considers the issues of efficiency and autonomy that are required to make reinforcement learning suitable for real-life control tasks. A real-time reinforcement learning algorithm is presented that repeatedly adjusts the control policy with the use of previously collected samples, and autonomously estimates the appropriate step-sizes for the learning updates. The algorithm is based on the actor-critic with experience replay whose step-sizes are determined on-line by an enhanced fixed point algorithm for on-line neural network training. An experimental study with simulated octopus arm and half-cheetah demonstrates the feasibility of the proposed algorithm to solve difficult learning control problems in an autonomous way within reasonably short time.
authors.affiliation: Warsaw University of Technology, Institute of Control and Computation Engineering, Poland. pwawrzynski@gmail.com
authors.forename: Paweł
authors.initials: P
authors.lastname: Wawrzyński
authors.forename: Ajay Kumar
authors.initials: AK
authors.lastname: Tanwani
mesh_terms: D000465:Algorithms; D000818:Animals; D001185:Artificial Intelligence; D003198:Computer Simulation; D008390:Markov Chains; D008962:Models, Theoretical; D009068:Movement; D016571:Neural Networks, Computer; D049831:Octopodiformes; D011340:Problem Solving; D012054:Reinforcement, Psychology; D013269:Stochastic Processes