title: Using automatically extracted information from mammography reports for decision-support.
abstract: To evaluate a system we developed that connects natural language processing (NLP) for information extraction from narrative text mammography reports with a Bayesian network for decision-support about breast cancer diagnosis. The ultimate goal of this system is to provide decision support as part of the workflow of producing the radiology report.
  
  
  
  We built a system that uses an NLP information extraction system (which extract BI-RADS descriptors and clinical information from mammography reports) to provide the necessary inputs to a Bayesian network (BN) decision support system (DSS) that estimates lesion malignancy from BI-RADS descriptors. We used this integrated system to predict diagnosis of breast cancer from radiology text reports and evaluated it with a reference standard of 300 mammography reports. We collected two different outputs from the DSS: (1) the probability of malignancy and (2) the BI-RADS final assessment category. Since NLP may produce imperfect inputs to the DSS, we compared the difference between using perfect ("reference standard") structured inputs to the DSS ("RS-DSS") vs NLP-derived inputs ("NLP-DSS") on the output of the DSS using the concordance correlation coefficient. We measured the classification accuracy of the BI-RADS final assessment category when using NLP-DSS, compared with the ground truth category established by the radiologist.
  
  
  
  The NLP-DSS and RS-DSS had closely matched probabilities, with a mean paired difference of 0.004±0.025. The concordance correlation of these paired measures was 0.95. The accuracy of the NLP-DSS to predict the correct BI-RADS final assessment category was 97.58%.
  
  
  
  The accuracy of the information extracted from mammography reports using the NLP system was sufficient to provide accurate DSS results. We believe our system could ultimately reduce the variation in practice in mammography related to assessment of malignant lesions and improve management decisions.
authors.affiliation: Akdeniz University Faculty of Medicine, Department of Biostatistics and Medical Informatics, Antalya, Turkey.
authors.forename: Selen
authors.initials: S
authors.lastname: Bozkurt
authors.affiliation: Department of Radiology and Medicine (Biomedical Informatics Research), Stanford University, Richard M. Lucas Center, 1201 Welch Road, Office P285, Stanford, CA 94305-5488, United States.
authors.forename: Francisco
authors.initials: F
authors.lastname: Gimenez
authors.affiliation: Department of Radiology, University of Wisconsin, Madison, WI, United States.
authors.forename: Elizabeth S
authors.initials: ES
authors.lastname: Burnside
authors.affiliation: Akdeniz University Faculty of Medicine, Department of Biostatistics and Medical Informatics, Antalya, Turkey.
authors.forename: Kemal H
authors.initials: KH
authors.lastname: Gulkesen
authors.affiliation: Department of Radiology and Medicine (Biomedical Informatics Research), Stanford University, Richard M. Lucas Center, 1201 Welch Road, Office P285, Stanford, CA 94305-5488, United States. Electronic address: dlrubin@stanford.edu.
authors.forename: Daniel L
authors.initials: DL
authors.lastname: Rubin
mesh_terms: D001499:Bayes Theorem; D001943:Breast Neoplasms / Q000000981:diagnostic imaging*; D020000:Decision Support Systems, Clinical; D005260:Female; D006801:Humans; D016247:Information Storage and Retrieval; D008327:Mammography / Q000706:statistics & numerical data*; D009323:Natural Language Processing
keywords: Breast Imaging Reporting and Data System (BI-RADS); Decision support systems; Information extraction; Natural language processing