title: Methodological Issues in Predicting Pediatric Epilepsy Surgery Candidates Through Natural Language Processing and Machine Learning.
abstract: We describe the development and evaluation of a system that uses machine learning and natural language processing techniques to identify potential candidates for surgical intervention for drug-resistant pediatric epilepsy. The data are comprised of free-text clinical notes extracted from the electronic health record (EHR). Both known clinical outcomes from the EHR and manual chart annotations provide gold standards for the patient's status. The following hypotheses are then tested: 1) machine learning methods can identify epilepsy surgery candidates as well as physicians do and 2) machine learning methods can identify candidates earlier than physicians do. These hypotheses are tested by systematically evaluating the effects of the data source, amount of training data, class balance, classification algorithm, and feature set on classifier performance. The results support both hypotheses, with F-measures ranging from 0.71 to 0.82. The feature set, classification algorithm, amount of training data, class balance, and gold standard all significantly affected classification performance. It was further observed that classification performance was better than the highest agreement between two annotators, even at one year before documented surgery referral. The results demonstrate that such machine learning methods can contribute to predicting pediatric epilepsy surgery candidates and reducing lag time to surgery referral.
authors.affiliation: Computational Bioscience Program, University of Colorado School of Medicine, Denver, CO, USA.
authors.forename: Kevin Bretonnel
authors.initials: KB
authors.lastname: Cohen
authors.affiliation: Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, OH, USA.
authors.forename: Benjamin
authors.initials: B
authors.lastname: Glass
authors.affiliation: Division of Neurology, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, OH, USA.
authors.forename: Hansel M
authors.initials: HM
authors.lastname: Greiner
authors.affiliation: Division of Neurology, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, OH, USA.
authors.forename: Katherine
authors.initials: K
authors.lastname: Holland-Bouley
authors.affiliation: Division of Neurology, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, OH, USA.
authors.forename: Shannon
authors.initials: S
authors.lastname: Standridge
authors.affiliation: Division of Neurology, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, OH, USA.
authors.forename: Ravindra
authors.initials: R
authors.lastname: Arya
authors.affiliation: Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, OH, USA.
authors.forename: Robert
authors.initials: R
authors.lastname: Faist
authors.affiliation: Division of Neurology, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, OH, USA.
authors.forename: Diego
authors.initials: D
authors.lastname: Morita
authors.affiliation: Division of Pediatric Neurosurgery, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, OH, USA.
authors.forename: Francesco
authors.initials: F
authors.lastname: Mangano
authors.affiliation: Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, OH, USA.
authors.forename: Brian
authors.initials: B
authors.lastname: Connolly
authors.affiliation: Division of Neurology, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, OH, USA.
authors.forename: Tracy
authors.initials: T
authors.lastname: Glauser
authors.affiliation: Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, University of Cincinnati, Cincinnati, OH, USA.
authors.forename: John
authors.initials: J
authors.lastname: Pestian
keywords: epilepsy; epilepsy surgery; machine learning; natural language processing; neurosurgery