title: Improving condition severity classification with an efficient active learning based framework.
abstract: Classification of condition severity can be useful for discriminating among sets of conditions or phenotypes, for example when prioritizing patient care or for other healthcare purposes. Electronic Health Records (EHRs) represent a rich source of labeled information that can be harnessed for severity classification. The labeling of EHRs is expensive and in many cases requires employing professionals with high level of expertise. In this study, we demonstrate the use of Active Learning (AL) techniques to decrease expert labeling efforts. We employ three AL methods and demonstrate their ability to reduce labeling efforts while effectively discriminating condition severity. We incorporate three AL methods into a new framework based on the original CAESAR (Classification Approach for Extracting Severity Automatically from Electronic Health Records) framework to create the Active Learning Enhancement framework (CAESAR-ALE). We applied CAESAR-ALE to a dataset containing 516 conditions of varying severity levels that were manually labeled by seven experts. Our dataset, called the "CAESAR dataset," was created from the medical records of 1.9 million patients treated at Columbia University Medical Center (CUMC). All three AL methods decreased labelers' efforts compared to the learning methods applied by the original CAESER framework in which the classifier was trained on the entire set of conditions; depending on the AL strategy used in the current study, the reduction ranged from 48% to 64% that can result in significant savings, both in time and money. As for the PPV (precision) measure, CAESAR-ALE achieved more than 13% absolute improvement in the predictive capabilities of the framework when classifying conditions as severe. These results demonstrate the potential of AL methods to decrease the labeling efforts of medical experts, while increasing accuracy given the same (or even a smaller) number of acquired conditions. We also demonstrated that the methods included in the CAESAR-ALE framework (Exploitation and Combination_XA) are more robust to the use of human labelers with different levels of professional expertise.
authors.affiliation: Information Systems Engineering, Ben-Gurion University of the Negev, Beer-Sheva, Israel; Malware Lab, Cyber Security Research Center, Ben-Gurion University of the Negev, Beer-Sheva, Israel. Electronic address: nirni@post.bgu.ac.il.
authors.forename: Nir
authors.initials: N
authors.lastname: Nissim
authors.affiliation: Department of Biomedical Informatics, Columbia University, New York, NY, USA; Observational Health Data Sciences and Informatics, Columbia University, New York, NY, USA.
authors.forename: Mary Regina
authors.initials: MR
authors.lastname: Boland
authors.affiliation: Department of Biomedical Informatics, Columbia University, New York, NY, USA; Department of Systems Biology, Columbia University, New York, NY, USA; Department of Medicine, Columbia University, New York, NY, USA; Observational Health Data Sciences and Informatics, Columbia University, New York, NY, USA.
authors.forename: Nicholas P
authors.initials: NP
authors.lastname: Tatonetti
authors.affiliation: Information Systems Engineering, Ben-Gurion University of the Negev, Beer-Sheva, Israel.
authors.forename: Yuval
authors.initials: Y
authors.lastname: Elovici
authors.affiliation: Department of Biomedical Informatics, Columbia University, New York, NY, USA; Observational Health Data Sciences and Informatics, Columbia University, New York, NY, USA.
authors.forename: George
authors.initials: G
authors.lastname: Hripcsak
authors.affiliation: Information Systems Engineering, Ben-Gurion University of the Negev, Beer-Sheva, Israel.
authors.forename: Yuval
authors.initials: Y
authors.lastname: Shahar
authors.affiliation: Department of Biomedical Informatics, Columbia University, New York, NY, USA; Department of Systems Biology, Columbia University, New York, NY, USA; Department of Medicine, Columbia University, New York, NY, USA; Observational Health Data Sciences and Informatics, Columbia University, New York, NY, USA. Electronic address: robert.moskovitch@columbia.edu.
authors.forename: Robert
authors.initials: R
authors.lastname: Moskovitch
mesh_terms: D000465:Algorithms; D001331:Automation; D066289:Data Curation; D057286:Electronic Health Records; D006801:Humans; D018794:Problem-Based Learning
keywords: Active learning; Condition; Electronic Health Records; Phenotyping; Severity