title: Penalized Q-Learning for Dynamic Treatment Regimens.
abstract: A dynamic treatment regimen incorporates both accrued information and long-term effects of treatment from specially designed clinical trials. As these trials become more and more popular in conjunction with longitudinal data from clinical studies, the development of statistical inference for optimal dynamic treatment regimens is a high priority. In this paper, we propose a new machine learning framework called penalized Q-learning, under which valid statistical inference is established. We also propose a new statistical procedure: individual selection and corresponding methods for incorporating individual selection within penalized Q-learning. Extensive numerical studies are presented which compare the proposed methods with existing methods, under a variety of scenarios, and demonstrate that the proposed approach is both inferentially and computationally superior. It is illustrated with a depression clinical trial study.
authors.affiliation: North Carolina State University, The University of Texas Health Science Center at Houston, and University of North Carolina.
authors.forename: R
authors.initials: R
authors.lastname: Song
authors.affiliation: North Carolina State University, The University of Texas Health Science Center at Houston, and University of North Carolina.
authors.forename: W
authors.initials: W
authors.lastname: Wang
authors.affiliation: North Carolina State University, The University of Texas Health Science Center at Houston, and University of North Carolina.
authors.forename: D
authors.initials: D
authors.lastname: Zeng
authors.affiliation: North Carolina State University, The University of Texas Health Science Center at Houston, and University of North Carolina.
authors.forename: M R
authors.initials: MR
authors.lastname: Kosorok
keywords: Dynamic treatment regimen; Individual selection; Multi-stage; Penalized Q-learning; Q-learning; Shrinkage; Two-stage procedure