title: Adapting machine learning techniques to censored time-to-event health record data: A general-purpose approach using inverse probability of censoring weighting.
abstract: Models for predicting the probability of experiencing various health outcomes or adverse events over a certain time frame (e.g., having a heart attack in the next 5years) based on individual patient characteristics are important tools for managing patient care. Electronic health data (EHD) are appealing sources of training data because they provide access to large amounts of rich individual-level data from present-day patient populations. However, because EHD are derived by extracting information from administrative and clinical databases, some fraction of subjects will not be under observation for the entire time frame over which one wants to make predictions; this loss to follow-up is often due to disenrollment from the health system. For subjects without complete follow-up, whether or not they experienced the adverse event is unknown, and in statistical terms the event time is said to be right-censored. Most machine learning approaches to the problem have been relatively ad hoc; for example, common approaches for handling observations in which the event status is unknown include (1) discarding those observations, (2) treating them as non-events, (3) splitting those observations into two observations: one where the event occurs and one where the event does not. In this paper, we present a general-purpose approach to account for right-censored outcomes using inverse probability of censoring weighting (IPCW). We illustrate how IPCW can easily be incorporated into a number of existing machine learning algorithms used to mine big health care data including Bayesian networks, k-nearest neighbors, decision trees, and generalized additive models. We then show that our approach leads to better calibrated predictions than the three ad hoc approaches when applied to predicting the 5-year risk of experiencing a cardiovascular adverse event, using EHD from a large U.S. Midwestern healthcare system.
authors.affiliation: Division of Biostatistics, School of Public Health, University of Minnesota, 420 Delaware Street S.E., MMC 303, Minneapolis, MN 55455, United States. Electronic address: vock@umn.edu.
authors.forename: David M
authors.initials: DM
authors.lastname: Vock
authors.affiliation: Division of Biostatistics, School of Public Health, University of Minnesota, 420 Delaware Street S.E., MMC 303, Minneapolis, MN 55455, United States. Electronic address: julianw@umn.edu.
authors.forename: Julian
authors.initials: J
authors.lastname: Wolfson
authors.affiliation: Department of Computer Science and Engineering, College of Science and Engineering, 200 Union Street, University of Minnesota, Minneapolis, MN 55455, United States. Electronic address: band0064@umn.edu.
authors.forename: Sunayan
authors.initials: S
authors.lastname: Bandyopadhyay
authors.affiliation: Department of Information and Decision Sciences, Carlson School of Management, University of Minnesota, 321 19th Avenue South, Minneapolis, MN 55455, United States. Electronic address: gedas@umn.edu.
authors.forename: Gediminas
authors.initials: G
authors.lastname: Adomavicius
authors.affiliation: Department of Information and Decision Sciences, Carlson School of Management, University of Minnesota, 321 19th Avenue South, Minneapolis, MN 55455, United States. Electronic address: johns021@umn.edu.
authors.forename: Paul E
authors.initials: PE
authors.lastname: Johnson
authors.affiliation: HealthPartners Institute for Education and Research, Mailstop 23301A, P.O. Box 1524, Minneapolis, MN 55440, United States. Electronic address: gabriela.x.vazquezbenitez@healthpartners.com.
authors.forename: Gabriela
authors.initials: G
authors.lastname: Vazquez-Benitez
authors.affiliation: HealthPartners Institute for Education and Research, Mailstop 23301A, P.O. Box 1524, Minneapolis, MN 55440, United States. Electronic address: patrick.j.oconnor@healthpartners.com.
authors.forename: Patrick J
authors.initials: PJ
authors.lastname: O'Connor
mesh_terms: D000465:Algorithms; D001499:Bayes Theorem; D016000:Cluster Analysis; D057286:Electronic Health Records; D006801:Humans; D000069550:Machine Learning; D011336:Probability
keywords: Censored data; Electronic health data; Inverse probability weighting; Machine learning; Risk prediction; Survival analysis