title: A boundedness result for the direct heuristic dynamic programming.
abstract: Approximate/adaptive dynamic programming (ADP) has been studied extensively in recent years for its potential scalability to solve large state and control space problems, including those involving continuous states and continuous controls. The applicability of ADP algorithms, especially the adaptive critic designs has been demonstrated in several case studies. Direct heuristic dynamic programming (direct HDP) is one of the ADP algorithms inspired by the adaptive critic designs. It has been shown applicable to industrial scale, realistic and complex control problems. In this paper, we provide a uniformly ultimately boundedness (UUB) result for the direct HDP learning controller under mild and intuitive conditions. By using a Lyapunov approach we show that the estimation errors of the learning parameters or the weights in the action and critic networks remain UUB. This result provides a useful controller convergence guarantee for the first time for the direct HDP design.
authors.affiliation: Department of Electrical Engineering, Tsinghua University, Beijing, 100084, PR China.
authors.forename: Feng
authors.initials: F
authors.lastname: Liu
authors.forename: Jian
authors.initials: J
authors.lastname: Sun
authors.forename: Jennie
authors.initials: J
authors.lastname: Si
authors.forename: Wentao
authors.initials: W
authors.lastname: Guo
authors.forename: Shengwei
authors.initials: S
authors.lastname: Mei
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D016571:Neural Networks, Computer; D009474:Neurons / Q000502:physiology; D009862:Online Systems; D011382:Programming, Linear