title: Applications of the self-organising map to reinforcement learning.
abstract: This article is concerned with the representation and generalisation of continuous action spaces in reinforcement learning (RL) problems. A model is proposed based on the self-organising map (SOM) of Kohonen [Self Organisation and Associative Memory, 1987] which allows either the one-to-one, many-to-one or one-to-many structure of the desired state-action mapping to be captured. Although presented here for tasks involving immediate reward, the approach is easily extended to delayed reward. We conclude that the SOM is a useful tool for providing real-time, on-line generalisation in RL problems in which the latent dimensionalities of the state and action spaces are small. Scalability issues are also discussed.
authors.affiliation: The Division of Informatics, Institute for Adaptive and Neural Computation, University of Edinburgh, UK. andys@dai.ed.ac.uk
authors.forename: Andrew James
authors.initials: AJ
authors.lastname: Smith
mesh_terms: D016571:Neural Networks, Computer; D011337:Probability Learning; D012054:Reinforcement, Psychology