title: Multi-Objective Markov Decision Processes for Data-Driven Decision Support.
abstract: We present new methodology based on Multi-Objective Markov Decision Processes for developing sequential decision support systems from data. Our approach uses sequential decision-making data to provide support that is useful to many different decision-makers, each with different, potentially time-varying preference. To accomplish this, we develop an extension of fitted-Qiteration for multiple objectives that computes policies for all scalarization functions, i.e. preference functions, simultaneously from continuous-state, finite-horizon data. We identify and address several conceptual and computational challenges along the way, and we introduce a new solution concept that is appropriate when different actions have similar expected outcomes. Finally, we demonstrate an application of our method using data from the Clinical Antipsychotic Trials of Intervention Effectiveness and show that our approach offers decision-makers increased choice by a larger class of optimal policies.
authors.affiliation: Department of Computer Science, Department of Epidemiology & Biostatistics, The University of Western Ontario, 1151 Richmond Street, London, ON N6A 3K7, Canada.
authors.forename: Daniel J
authors.initials: DJ
authors.lastname: Lizotte
authors.affiliation: Department of Statistics, North Carolina State University, Raliegh, NC 27695, USA.
authors.forename: Eric B
authors.initials: EB
authors.lastname: Laber
keywords: Markov decision processes; clinical decision support; evidence-based medicine; multi-objective optimization; reinforcement learning