title: Reward-weighted regression with sample reuse for direct policy search in reinforcement learning.
abstract: Direct policy search is a promising reinforcement learning framework, in particular for controlling continuous, high-dimensional systems. Policy search often requires a large number of samples for obtaining a stable policy update estimator, and this is prohibitive when the sampling cost is expensive. In this letter, we extend an expectation-maximization-based policy search method so that previously collected samples can be efficiently reused. The usefulness of the proposed method, reward-weighted regression with sample reuse (R3), is demonstrated through robot learning experiments. (This letter is an extended version of our earlier conference paper: Hachiya, Peters, & Sugiyama, 2009 .).
authors.affiliation: Tokyo Institute of Technology, O-okayama, Meguro-ku, Tokyo 152-8552, Japan. hachiya@sg.cs.titech.ac.jp
authors.forename: Hirotaka
authors.initials: H
authors.lastname: Hachiya
authors.forename: Jan
authors.initials: J
authors.lastname: Peters
authors.forename: Masashi
authors.initials: M
authors.lastname: Sugiyama
mesh_terms: D000818:Animals; D001185:Artificial Intelligence; D006801:Humans; D007858:Learning / Q000502:physiology*; D016571:Neural Networks, Computer; D012054:Reinforcement, Psychology; D012201:Reward