title: New Statistical Learning Methods for Estimating Optimal Dynamic Treatment Regimes.
abstract: Dynamic treatment regimes (DTRs) are sequential decision rules for individual patients that can adapt over time to an evolving illness. The goal is to accommodate heterogeneity among patients and find the DTR which will produce the best long term outcome if implemented. We introduce two new statistical learning methods for estimating the optimal DTR, termed backward outcome weighted learning (BOWL), and simultaneous outcome weighted learning (SOWL). These approaches convert individualized treatment selection into an either sequential or simultaneous classification problem, and can thus be applied by modifying existing machine learning techniques. The proposed methods are based on directly maximizing over all DTRs a nonparametric estimator of the expected long-term outcome; this is fundamentally different than regression-based methods, for exampleQ-learning, which indirectly attempt such maximization and rely heavily on the correctness of postulated regression models. We prove that the resulting rules are consistent, and provide finite sample bounds for the errors using the estimated rules. Simulation results suggest the proposed methods produce superior DTRs compared withQ-learning especially in small samples. We illustrate the methods using data from a clinical trial for smoking cessation.
authors.affiliation: Assistant Professor, Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, WI 53792.
authors.forename: Ying-Qi
authors.initials: YQ
authors.lastname: Zhao
authors.affiliation: Professor, Department of Biostatistics, University of North Carolina at Chapel Hill, NC 27599.
authors.forename: Donglin
authors.initials: D
authors.lastname: Zeng
authors.affiliation: Assistant Professor, Department of Statistics, North Carolina State University, NC 27695.
authors.forename: Eric B
authors.initials: EB
authors.lastname: Laber
authors.affiliation: W. R. Kenan, Jr. Distiguished Professor and Chair, Department of Biostatistics, and Professor, Department of Statistics and Operations Research, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599.
authors.forename: Michael R
authors.initials: MR
authors.lastname: Kosorok
keywords: Classification; Dynamic treatment regimes; Personalized medicine; Q-learning; Reinforcement learning; Risk Bound; Support vector machine