Applications of the self-organising map to reinforcement learning.
This article is concerned with the representation and generalisation of continuous action spaces in reinforcement learning (RL) problems. A model is proposed based on the self-organising map (SOM) of Kohonen [Self Organisation and Associative Memory, 1987] which allows either the one-to-one, many-to-one or one-to-many structure of the desired state-action mapping to be captured. Although presented here for tasks involving immediate reward, the approach is easily extended to delayed reward. We conclude that the SOM is a useful tool for providing real-time, on-line generalisation in RL problems in which the latent dimensionalities of the state and action spaces are small. Scalability issues are also discussed.
Smith
Andrew James
AJ
The Division of Informatics, Institute for Adaptive and Neural Computation, University of Edinburgh, UK. andys@dai.ed.ac.uk
D016571:Neural Networks, Computer; D011337:Probability Learning; D012054:Reinforcement, Psychology