Positive impact of state similarity on reinforcement learning performance.
In this paper, we propose a novel approach to identify states with similar subpolicies and show how they can be integrated into the reinforcement learning framework to improve learning performance. The method utilizes a specialized tree structure to identify common action sequences of states, which are derived from possible optimal policies, and defines a similarity function between two states based on the number of such sequences. Using this similarity function, updates on the action-value function of a state are reflected onto all similar states. This allows experience that is acquired during learning to be applied to a broader context. The effectiveness of the method is demonstrated empirically.
Girgin
Sertan
S
Department of Computer Engineering, Middle East Technical University, 06531 Ankara, Turkey.
Polat
Faruk
F
Alhajj
Reda
R
D000465:Algorithms; D001185:Artificial Intelligence; D003198:Computer Simulation; D008962:Models, Theoretical; D010363:Pattern Recognition, Automated / Q000379:methods*