title: Model-based reinforcement learning with dimension reduction.
abstract: The goal of reinforcement learning is to learn an optimal policy which controls an agent to acquire the maximum cumulative reward. The model-based reinforcement learning approach learns a transition model of the environment from data, and then derives the optimal policy using the transition model. However, learning an accurate transition model in high-dimensional environments requires a large amount of data which is difficult to obtain. To overcome this difficulty, in this paper, we propose to combine model-based reinforcement learning with the recently developed least-squares conditional entropy (LSCE) method, which simultaneously performs transition model estimation and dimension reduction. We also further extend the proposed method to imitation learning scenarios. The experimental results show that policy search combined with LSCE performs well for high-dimensional control tasks including real humanoid robot control.
authors.affiliation: Department of Computer Science, The University of Tokyo, Japan. Electronic address: voot@ms.k.u-tokyo.ac.jp.
authors.forename: Voot
authors.initials: V
authors.lastname: Tangkaratt
authors.affiliation: Department of Brain Robot Interface, ATR Computational Neuroscience Laboratory, Japan. Electronic address: xmorimo@atr.jp.
authors.forename: Jun
authors.initials: J
authors.lastname: Morimoto
authors.affiliation: Department of Complexity Science and Engineering, The University of Tokyo, Japan. Electronic address: sugi@k.u-tokyo.ac.jp.
authors.forename: Masashi
authors.initials: M
authors.lastname: Sugiyama
mesh_terms: D001185:Artificial Intelligence / Q000639:trends; D019277:Entropy; D007858:Learning; D016018:Least-Squares Analysis; D000069550:Machine Learning / Q000639:trends; D008962:Models, Theoretical; D012054:Reinforcement, Psychology; D012201:Reward
keywords: Model-based reinforcement learning; Sufficient dimension reduction; Transition model estimation