title: Context transfer in reinforcement learning using action-value functions.
abstract: This paper discusses the notion of context transfer in reinforcement learning tasks. Context transfer, as defined in this paper, implies knowledge transfer between source and target tasks that share the same environment dynamics and reward function but have different states or action spaces. In other words, the agents learn the same task while using different sensors and actuators. This requires the existence of an underlying common Markov decision process (MDP) to which all the agents' MDPs can be mapped. This is formulated in terms of the notion of MDP homomorphism. The learning framework is Q-learning. To transfer the knowledge between these tasks, the feature space is used as a translator and is expressed as a partial mapping between the state-action spaces of different tasks. The Q-values learned during the learning process of the source tasks are mapped to the sets of Q-values for the target task. These transferred Q-values are merged together and used to initialize the learning process of the target task. An interval-based approach is used to represent and merge the knowledge of the source tasks. Empirical results show that the transferred initialization can be beneficial to the learning process of the target task.
authors.affiliation: Cognitive Robotics Lab, Control and Intelligent Processing Center of Excellence, School of Electrical and Computer Engineering, College of Engineering, University of Tehran, P.O. Box 14395-515, Tehran, Iran.
authors.forename: Amin
authors.initials: A
authors.lastname: Mousavi
authors.affiliation: Cognitive Robotics Lab, Control and Intelligent Processing Center of Excellence, School of Electrical and Computer Engineering, College of Engineering, University of Tehran, P.O. Box 14395-515, Tehran, Iran ; School of Cognitive Science, Institute for Research in Fundamental Sciences (IPM), P.O. Box 19395-5746, Tehran, Iran.
authors.forename: Babak
authors.initials: B
authors.lastname: Nadjar Araabi
authors.affiliation: Cognitive Robotics Lab, Control and Intelligent Processing Center of Excellence, School of Electrical and Computer Engineering, College of Engineering, University of Tehran, P.O. Box 14395-515, Tehran, Iran ; School of Cognitive Science, Institute for Research in Fundamental Sciences (IPM), P.O. Box 19395-5746, Tehran, Iran.
authors.forename: Majid
authors.initials: M
authors.lastname: Nili Ahmadabadi
mesh_terms: D001185:Artificial Intelligence; D002755:Choice Behavior; D004777:Environment; D006801:Humans; D007721:Knowledge of Results, Psychological; D008390:Markov Chains; D008960:Models, Psychological; D012054:Reinforcement, Psychology; D014163:Transfer, Psychology