Self-organizing neural architectures and cooperative learning in a multiagent environment.
Temporal-Difference-Fusion Architecture for Learning, Cognition, and Navigation (TD-FALCON) is a generalization of adaptive resonance theory (a class of self-organizing neural networks) that incorporates TD methods for real-time reinforcement learning. In this paper, we investigate how a team of TD-FALCON networks may cooperate to learn and function in a dynamic multiagent environment based on minefield navigation and a predator/prey pursuit tasks. Experiments on the navigation task demonstrate that TD-FALCON agent teams are able to adapt and function well in a multiagent environment without an explicit mechanism of collaboration. In comparison, traditional Q-learning agents using gradient-descent-based feedforward neural networks, trained with the standard backpropagation and the resilient-propagation (RPROP) algorithms, produce a significantly poorer level of performance. For the predator/prey pursuit task, we experiment with various cooperative strategies and find that a combination of a high-level compressed state representation and a hybrid reward function produces the best results. Using the same cooperative strategy, the TD-FALCON team also outperforms the RPROP-based reinforcement learners in terms of both task completion rate and learning efficiency.
Xiao
Dan
D
School of Computer Engineering, Nanyang Technological University, Singapore 639798, Singapore. xiao0002@ntu.edu.sg
Tan
Ah-Hwee
AH
D000465:Algorithms; D003198:Computer Simulation; D003661:Decision Support Techniques; D008962:Models, Theoretical; D016571:Neural Networks, Computer; D010363:Pattern Recognition, Automated / Q000379:methods*