title: Reinforcement learning strategies for clinical trials in nonsmall cell lung cancer.
abstract: Typical regimens for advanced metastatic stage IIIB/IV nonsmall cell lung cancer (NSCLC) consist of multiple lines of treatment. We present an adaptive reinforcement learning approach to discover optimal individualized treatment regimens from a specially designed clinical trial (a "clinical reinforcement trial") of an experimental treatment for patients with advanced NSCLC who have not been treated previously with systemic therapy. In addition to the complexity of the problem of selecting optimal compounds for first- and second-line treatments based on prognostic factors, another primary goal is to determine the optimal time to initiate second-line therapy, either immediately or delayed after induction therapy, yielding the longest overall survival time. A reinforcement learning method called Q-learning is utilized, which involves learning an optimal regimen from patient data generated from the clinical reinforcement trial. Approximating the Q-function with time-indexed parameters can be achieved by using a modification of support vector regression that can utilize censored data. Within this framework, a simulation study shows that the procedure can extract optimal regimens for two lines of treatment directly from clinical data without prior knowledge of the treatment effect mechanism. In addition, we demonstrate that the design reliably selects the best initial time for second-line therapy while taking into account the heterogeneity of NSCLC across patients.
authors.affiliation: Global Biostatistics and Epidemiology, Amgen Inc., One Amgen Center Drive, Thousand Oaks, California 91320, USA.
authors.forename: Yufan
authors.initials: Y
authors.lastname: Zhao
authors.forename: Donglin
authors.initials: D
authors.lastname: Zeng
authors.forename: Mark A
authors.initials: MA
authors.lastname: Socinski
authors.forename: Michael R
authors.initials: MR
authors.lastname: Kosorok
mesh_terms: D000970:Antineoplastic Agents / Q000627:therapeutic use*; D001185:Artificial Intelligence; D002289:Carcinoma, Non-Small-Cell Lung / Q000175:diagnosis / Q000188:drug therapy* / Q000453:epidemiology; D002986:Clinical Trials as Topic / Q000379:methods*; D003627:Data Interpretation, Statistical; D004360:Drug Therapy, Computer-Assisted / Q000379:methods* / Q000706:statistics & numerical data; D006801:Humans; D008175:Lung Neoplasms / Q000188:drug therapy* / Q000453:epidemiology; D017063:Outcome Assessment, Health Care / Q000379:methods* / Q000706:statistics & numerical data; D011379:Prognosis; D012054:Reinforcement, Psychology; D016896:Treatment Outcome
chemical_list: D000970:Antineoplastic Agents