title: Reinforcement-learning-based output-feedback control of nonstrict nonlinear discrete-time systems with application to engine emission control.
abstract: A novel reinforcement-learning-based output adaptive neural network (NN) controller, which is also referred to as the adaptive-critic NN controller, is developed to deliver the desired tracking performance for a class of nonlinear discrete-time systems expressed in nonstrict feedback form in the presence of bounded and unknown disturbances. The adaptive-critic NN controller consists of an observer, a critic, and two action NNs. The observer estimates the states and output, and the two action NNs provide virtual and actual control inputs to the nonlinear discrete-time system. The critic approximates a certain strategic utility function, and the action NNs minimize the strategic utility function and control inputs. All NN weights adapt online toward minimization of a performance index, utilizing the gradient-descent-based rule, in contrast with iteration-based adaptive-critic schemes. Lyapunov functions are used to show the stability of the closed-loop tracking error, weights, and observer estimates. Separation and certainty equivalence principles, persistency of excitation condition, and linearity in the unknown parameter assumption are not needed. Experimental results on a spark ignition (SI) engine operating lean at an equivalence ratio of 0.75 show a significant (25%) reduction in cyclic dispersion in heat release with control, while the average fuel input changes by less than 1% compared with the uncontrolled case. Consequently, oxides of nitrogen (NO(x)) drop by 30%, and unburned hydrocarbons drop by 16% with control. Overall, NO(x)'s are reduced by over 80% compared with stoichiometric levels.
authors.affiliation: Department of Electrical and Computer Engineering, Missouri University of Science and Technology, Rolla, MO 65409, USA.
authors.forename: Peter
authors.initials: P
authors.lastname: Shih
authors.forename: Brian C
authors.initials: BC
authors.lastname: Kaul
authors.forename: Sarangapani
authors.initials: S
authors.lastname: Jagannathan
authors.forename: James A
authors.initials: JA
authors.lastname: Drallmeier