title: Generating adaptive behaviour within a memory-prediction framework.
abstract: The Memory-Prediction Framework (MPF) and its Hierarchical-Temporal Memory implementation (HTM) have been widely applied to unsupervised learning problems, for both classification and prediction. To date, there has been no attempt to incorporate MPF/HTM in reinforcement learning or other adaptive systems; that is, to use knowledge embodied within the hierarchy to control a system, or to generate behaviour for an agent. This problem is interesting because the human neocortex is believed to play a vital role in the generation of behaviour, and the MPF is a model of the human neocortex.We propose some simple and biologically-plausible enhancements to the Memory-Prediction Framework. These cause it to explore and interact with an external world, while trying to maximize a continuous, time-varying reward function. All behaviour is generated and controlled within the MPF hierarchy. The hierarchy develops from a random initial configuration by interaction with the world and reinforcement learning only. Among other demonstrations, we show that a 2-node hierarchy can learn to successfully play "rocks, paper, scissors" against a predictable opponent.
authors.affiliation: Department of Electrical and Electronic Engineering, National ICT Australia, University of Melbourne, Victoria, Australia. david.rawlinson@nicta.com.au
authors.forename: David
authors.initials: D
authors.lastname: Rawlinson
authors.forename: Gideon
authors.initials: G
authors.lastname: Kowadlo
mesh_terms: D000223:Adaptation, Psychological / Q000502:physiology*; D000465:Algorithms; D001519:Behavior / Q000502:physiology*; D006801:Humans; D007858:Learning / Q000502:physiology; D008568:Memory / Q000502:physiology*; D008959:Models, Neurological; D008960:Models, Psychological; D019579:Neocortex / Q000502:physiology; D010364:Pattern Recognition, Visual / Q000502:physiology; D012054:Reinforcement, Psychology; D012201:Reward; D014584:User-Computer Interface