title: Extractive text summarization system to aid data extraction from full text in systematic review development.
abstract: Extracting data from publication reports is a standard process in systematic review (SR) development. However, the data extraction process still relies too much on manual effort which is slow, costly, and subject to human error. In this study, we developed a text summarization system aimed at enhancing productivity and reducing errors in the traditional data extraction process.
  
  
  
  We developed a computer system that used machine learning and natural language processing approaches to automatically generate summaries of full-text scientific publications. The summaries at the sentence and fragment levels were evaluated in finding common clinical SR data elements such as sample size, group size, and PICO values. We compared the computer-generated summaries with human written summaries (title and abstract) in terms of the presence of necessary information for the data extraction as presented in the Cochrane review's study characteristics tables.
  
  
  
  At the sentence level, the computer-generated summaries covered more information than humans do for systematic reviews (recall 91.2% vs. 83.8%, p<0.001). They also had a better density of relevant sentences (precision 59% vs. 39%, p<0.001). At the fragment level, the ensemble approach combining rule-based, concept mapping, and dictionary-based methods performed better than individual methods alone, achieving an 84.7% F-measure.
  
  
  
  Computer-generated summaries are potential alternative information sources for data extraction in systematic review development. Machine learning and natural language processing are promising approaches to the development of such an extractive summarization system.
authors.affiliation: Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA; Division of Health and Biomedical Informatics, Northwestern University, Chicago, IL, USA. Electronic address: duy.bui@utah.edu.
authors.forename: Duy Duc An
authors.initials: DDA
authors.lastname: Bui
authors.affiliation: Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA.
authors.forename: Guilherme
authors.initials: G
authors.lastname: Del Fiol
authors.affiliation: Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA.
authors.forename: John F
authors.initials: JF
authors.lastname: Hurdle
authors.affiliation: Division of Health and Biomedical Informatics, Northwestern University, Chicago, IL, USA.
authors.forename: Siddhartha
authors.initials: S
authors.lastname: Jonnalagadda
mesh_terms: D006801:Humans; D057225:Data Mining; D007802:Language; D000069550:Machine Learning; D009323:Natural Language Processing; D011642:Publications; D000078202:Systematic Reviews as Topic
keywords: Data collection; Machine learning; Review literature as topic; Text classification; Text summarization