title: Q-learning residual analysis: application to the effectiveness of sequences of antipsychotic medications for patients with schizophrenia.
abstract: Q-learning is a regression-based approach that uses longitudinal data to construct dynamic treatment regimes, which are sequences of decision rules that use patient information to inform future treatment decisions. An optimal dynamic treatment regime is composed of a sequence of decision rules that indicate how to optimally individualize treatment using the patients' baseline and time-varying characteristics to optimize the final outcome. Constructing optimal dynamic regimes using Q-learning depends heavily on the assumption that regression models at each decision point are correctly specified; yet model checking in the context of Q-learning has been largely overlooked in the current literature. In this article, we show that residual plots obtained from standard Q-learning models may fail to adequately check the quality of the model fit. We present a modified Q-learning procedure that accommodates residual analyses using standard tools. We present simulation studies showing the advantage of the proposed modification over standard Q-learning. We illustrate this new Q-learning approach using data collected from a sequential multiple assignment randomized trial of patients with schizophrenia. Copyright © 2016 John Wiley & Sons, Ltd.
authors.affiliation: Department of Statistics, University of Pennsylvania, Philadelphia, PA, U.S.A.|Center for Pharmacoepidemiology Research and Training, University of Pennsylvania, Philadelphia, PA, U.S.A.
authors.forename: Ashkan
authors.initials: A
authors.lastname: Ertefaie
authors.affiliation: Biostatistics Unit, GroupHealth Research Institute, Seattle, WA, U.S.A.
authors.forename: Susan
authors.initials: S
authors.lastname: Shortreed
authors.affiliation: Centre for Quantitative Medicine, Duke-NUS Graduate Medical School, Singapore.|Department of Biostatistics, Columbia University, New York, NY, U.S.A.
authors.forename: Bibhas
authors.initials: B
authors.lastname: Chakraborty
mesh_terms: D014150:Antipsychotic Agents / Q000008:administration & dosage* / Q000627:therapeutic use; D003627:Data Interpretation, Statistical; D003661:Decision Support Techniques; D004334:Drug Administration Schedule; D006801:Humans; D016014:Linear Models; D015233:Models, Statistical; D012044:Regression Analysis; D012559:Schizophrenia / Q000188:drug therapy*; D016896:Treatment Outcome
chemical_list: D014150:Antipsychotic Agents
keywords: Q-learning; Residual analysis; SMART designs; dynamic treatment regimes