title: Continuous-time Q-learning for infinite-horizon discounted cost linear quadratic regulator problems.
abstract: This paper presents a method of Q-learning to solve the discounted linear quadratic regulator (LQR) problem for continuous-time (CT) continuous-state systems. Most available methods in the existing literature for CT systems to solve the LQR problem generally need partial or complete knowledge of the system dynamics. Q-learning is effective for unknown dynamical systems, but has generally been well understood only for discrete-time systems. The contribution of this paper is to present a Q-learning methodology for CT systems which solves the LQR problem without having any knowledge of the system dynamics. A natural and rigorous justified parameterization of the Q-function is given in terms of the state, the control input, and its derivatives. This parameterization allows the implementation of an online Q-learning algorithm for CT systems. The simulation results supporting the theoretical development are also presented.
authors.forename: Muthukumar
authors.initials: M
authors.lastname: Palanisamy
authors.forename: Hamidreza
authors.initials: H
authors.lastname: Modares
authors.forename: Frank L
authors.initials: FL
authors.lastname: Lewis
authors.forename: Muhammad
authors.initials: M
authors.lastname: Aurangzeb