Robust reinforcement learning control using integral quadratic constraints for recurrent neural networks.
The applicability of machine learning techniques for feedback control systems is limited by a lack of stability guarantees. Robust control theory offers a framework for analyzing the stability of feedback control loops, but for the integral quadratic constraint (IQC) framework used here, all components are required to be represented as linear, time-invariant systems plus uncertainties with, for IQCs used here, bounded gain. In this paper, the stability of a control loop including a recurrent neural network (NN) is analyzed by replacing the nonlinear and time-varying components of the NN with IQCs on their gain. As a result, a range of the NN's weights is found within which stability is guaranteed. An algorithm is demonstrated for training the recurrent NN using reinforcement learning and guaranteeing stability while learning.
Department of Computer Science, Colorado State University, Fort Collins, CO 80523-1873, USA. anderson@cs.colostate.edu
Charles W
CW
Anderson
Peter Michael
PM
Young
Michael R
MR
Buehner
James N
JN
Knight
Keith A
KA
Bush
Douglas C
DC
Hittle
D000465:Algorithms; D001185:Artificial Intelligence; D032701:Biomimetics / Q000379:methods*; D003198:Computer Simulation; D003661:Decision Support Techniques; D005246:Feedback; D016018:Least-Squares Analysis; D008390:Markov Chains; D008962:Models, Theoretical; D016571:Neural Networks, Computer; D012054:Reinforcement, Psychology