Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems.
In this paper we present in a continuous-time framework an online approach to direct adaptive optimal control with infinite horizon cost for nonlinear systems. The algorithm converges online to the optimal control solution without knowledge of the internal system dynamics. Closed-loop dynamic stability is guaranteed throughout. The algorithm is based on a reinforcement learning scheme, namely Policy Iterations, and makes use of neural networks, in an Actor/Critic structure, to parametrically represent the control policy and the performance of the control system. The two neural networks are trained to express the optimal controller and optimal cost function which describes the infinite horizon control performance. Convergence of the algorithm is proven under the realistic assumption that the two neural networks do not provide perfect representations for the nonlinear control and cost functions. The result is a hybrid control structure which involves a continuous-time controller and a supervisory adaptation structure which operates based on data sampled from the plant and from the continuous-time performance dynamics. Such control structure is unlike any standard form of controllers previously seen in the literature. Simulation results, obtained considering two second-order nonlinear systems, are provided.
Automation and Robotics Research Institute, University of Texas at Arlington, 7300 Jack Newell Blvd. S., Fort Worth, TX 76118, USA. dvrabie@uta.edu
Draguna
D
Vrabie
Frank
F
Lewis
D000223:Adaptation, Psychological; D000465:Algorithms; D003198:Computer Simulation; D003661:Decision Support Techniques; D017143:Fuzzy Logic; D016571:Neural Networks, Computer; D009483:Neuropsychological Tests; D017711:Nonlinear Dynamics; D010363:Pattern Recognition, Automated; D012815:Signal Processing, Computer-Assisted; D013598:Systems Theory