title: Learning regular expressions for clinical text classification.
abstract: OBJECTIVE
  Natural language processing (NLP) applications typically use regular expressions that have been developed manually by human experts. Our goal is to automate both the creation and utilization of regular expressions in text classification.
  
  
  METHODS
  We designed a novel regular expression discovery (RED) algorithm and implemented two text classifiers based on RED. The RED+ALIGN classifier combines RED with an alignment algorithm, and RED+SVM combines RED with a support vector machine (SVM) classifier. Two clinical datasets were used for testing and evaluation: the SMOKE dataset, containing 1091 text snippets describing smoking status; and the PAIN dataset, containing 702 snippets describing pain status. We performed 10-fold cross-validation to calculate accuracy, precision, recall, and F-measure metrics. In the evaluation, an SVM classifier was trained as the control.
  
  
  RESULTS
  The two RED classifiers achieved 80.9-83.0% in overall accuracy on the two datasets, which is 1.3-3% higher than SVM's accuracy (p<0.001). Similarly, small but consistent improvements have been observed in precision, recall, and F-measure when RED classifiers are compared with SVM alone. More significantly, RED+ALIGN correctly classified many instances that were misclassified by the SVM classifier (8.1-10.3% of the total instances and 43.8-53.0% of SVM's misclassifications).
  
  
  CONCLUSIONS
  Machine-generated regular expressions can be effectively used in clinical text classification. The regular expression-based classifier can be combined with other classifiers, like SVM, to improve classification performance.
authors.affiliation: Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA VA Salt Lake City Health Care System, Salt Lake City, Utah, USA.
authors.forename: Duy Duc An
authors.initials: DD
authors.lastname: Bui
authors.affiliation: Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA VA Salt Lake City Health Care System, Salt Lake City, Utah, USA.
authors.forename: Qing
authors.initials: Q
authors.lastname: Zeng-Treitler
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D001330:Electronic Data Processing; D006801:Humans; D016347:Medical Records Systems, Computerized / Q000145:classification*; D009323:Natural Language Processing; D010146:Pain / Q000145:classification; D012907:Smoking; D060388:Support Vector Machine
keywords: Machine Learning; Natural Language Processing; Regular Expressions; Support Vector Machines; Text Classification