title: Incremental learning of skill collections based on intrinsic motivation.
abstract: Life-long learning of reusable, versatile skills is a key prerequisite for embodied agents that act in a complex, dynamic environment and are faced with different tasks over their lifetime. We address the question of how an agent can learn useful skills efficiently during a developmental period, i.e., when no task is imposed on him and no external reward signal is provided. Learning of skills in a developmental period needs to be incremental and self-motivated. We propose a new incremental, task-independent skill discovery approach that is suited for continuous domains. Furthermore, the agent learns specific skills based on intrinsic motivation mechanisms that determine on which skills learning is focused at a given point in time. We evaluate the approach in a reinforcement learning setup in two continuous domains with complex dynamics. We show that an intrinsically motivated, skill learning agent outperforms an agent which learns task solutions from scratch. Furthermore, we compare different intrinsic motivation mechanisms and how efficiently they make use of the agent's developmental period.
authors.affiliation: Robotics Research Group, Faculty 3 - Mathematics and Computer Science, Universität Bremen Bremen, Germany.
authors.forename: Jan H
authors.initials: JH
authors.lastname: Metzen
authors.forename: Frank
authors.initials: F
authors.lastname: Kirchner
keywords: graph-based representation; hierarchical reinforcement learning; intrinsic motivation; life-long learning; skill discovery
references.citation: Science. 2000 Dec 22;290(5500):2319-23
references.pmid: 11125149
references.citation: J Comp Physiol Psychol. 1950 Aug;43(4):289-94
references.pmid: 15436888
references.citation: J Neurophysiol. 2005 Aug;94(2):1443-58
references.pmid: 15829594
references.citation: Front Neurorobot. 2007 Nov 02;1:6
references.pmid: 18958277