title: Efficient model learning methods for actor-critic control.
abstract: We propose two new actor-critic algorithms for reinforcement learning. Both algorithms use local linear regression (LLR) to learn approximations of the functions involved. A crucial feature of the algorithms is that they also learn a process model, and this, in combination with LLR, provides an efficient policy update for faster learning. The first algorithm uses a novel model-based update rule for the actor parameters. The second algorithm does not use an explicit actor but learns a reference model which represents a desired behavior, from which desired control actions can be calculated using the inverse of the learned process model. The two novel methods and a standard actor-critic algorithm are applied to the pendulum swing-up problem, in which the novel methods achieve faster learning than the standard algorithm.
authors.affiliation: Delft Center for Systems and Control, Delft University of Technology, Delft, The Netherlands. i.grondman@tudelft.nl
authors.forename: Ivo
authors.initials: I
authors.lastname: Grondman
authors.forename: Maarten
authors.initials: M
authors.lastname: Vaandrager
authors.forename: Lucian
authors.initials: L
authors.lastname: Buşoniu
authors.forename: Robert
authors.initials: R
authors.lastname: Babuska
authors.forename: Erik
authors.initials: E
authors.lastname: Schuitema
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D003198:Computer Simulation; D003661:Decision Support Techniques; D008962:Models, Theoretical; D010363:Pattern Recognition, Automated / Q000379:methods*