title: SWIFT-Review: a text-mining workbench for systematic review.
abstract: BACKGROUND
  There is growing interest in using machine learning approaches to priority rank studies and reduce human burden in screening literature when conducting systematic reviews. In addition, identifying addressable questions during the problem formulation phase of systematic review can be challenging, especially for topics having a large literature base. Here, we assess the performance of the SWIFT-Review priority ranking algorithm for identifying studies relevant to a given research question. We also explore the use of SWIFT-Review during problem formulation to identify, categorize, and visualize research areas that are data rich/data poor within a large literature corpus.
  
  
  METHODS
  Twenty case studies, including 15 public data sets, representing a range of complexity and size, were used to assess the priority ranking performance of SWIFT-Review. For each study, seed sets of manually annotated included and excluded titles and abstracts were used for machine training. The remaining references were then ranked for relevance using an algorithm that considers term frequency and latent Dirichlet allocation (LDA) topic modeling. This ranking was evaluated with respect to (1) the number of studies screened in order to identify 95 % of known relevant studies and (2) the "Work Saved over Sampling" (WSS) performance metric. To assess SWIFT-Review for use in problem formulation, PubMed literature search results for 171 chemicals implicated as EDCs were uploaded into SWIFT-Review (264,588 studies) and categorized based on evidence stream and health outcome. Patterns of search results were surveyed and visualized using a variety of interactive graphics.
  
  
  RESULTS
  Compared with the reported performance of other tools using the same datasets, the SWIFT-Review ranking procedure obtained the highest scores on 11 out of 15 of the public datasets. Overall, these results suggest that using machine learning to triage documents for screening has the potential to save, on average, more than 50 % of the screening effort ordinarily required when using un-ordered document lists. In addition, the tagging and annotation capabilities of SWIFT-Review can be useful during the activities of scoping and problem formulation.
  
  
  CONCLUSIONS
  Text-mining and machine learning software such as SWIFT-Review can be valuable tools to reduce the human screening burden and assist in problem formulation.
authors.affiliation: SciOme LLC, Research Triangle Park, 2 Davis Drive, 27709, NC, USA. brian.howard@sciome.com.
authors.forename: Brian E
authors.initials: BE
authors.lastname: Howard
authors.affiliation: SciOme LLC, Research Triangle Park, 2 Davis Drive, 27709, NC, USA.
authors.forename: Jason
authors.initials: J
authors.lastname: Phillips
authors.affiliation: SciOme LLC, Research Triangle Park, 2 Davis Drive, 27709, NC, USA.
authors.forename: Kyle
authors.initials: K
authors.lastname: Miller
authors.affiliation: SciOme LLC, Research Triangle Park, 2 Davis Drive, 27709, NC, USA.
authors.forename: Arpit
authors.initials: A
authors.lastname: Tandon
authors.affiliation: SciOme LLC, Research Triangle Park, 2 Davis Drive, 27709, NC, USA.
authors.forename: Deepak
authors.initials: D
authors.lastname: Mav
authors.affiliation: SciOme LLC, Research Triangle Park, 2 Davis Drive, 27709, NC, USA.
authors.forename: Mihir R
authors.initials: MR
authors.lastname: Shah
authors.affiliation: Office of Scientific Information Management, National Institute of Environmental Health Sciences, National Institutes of Health, Department of Health and Human Services, Research Triangle Park, NC, USA.
authors.forename: Stephanie
authors.initials: S
authors.lastname: Holmgren
authors.affiliation: Division of the National Toxicology Program, National Institute of Environmental Health Sciences, National Institutes of Health, Department of Health and Human Services, Research Triangle Park, NC, USA.
authors.forename: Katherine E
authors.initials: KE
authors.lastname: Pelch
authors.affiliation: Division of the National Toxicology Program, National Institute of Environmental Health Sciences, National Institutes of Health, Department of Health and Human Services, Research Triangle Park, NC, USA.
authors.forename: Vickie
authors.initials: V
authors.lastname: Walker
authors.affiliation: Division of the National Toxicology Program, National Institute of Environmental Health Sciences, National Institutes of Health, Department of Health and Human Services, Research Triangle Park, NC, USA.
authors.forename: Andrew A
authors.initials: AA
authors.lastname: Rooney
authors.affiliation: Centre for Clinical Brain Sciences, University of Edinburgh, Scotland, UK.
authors.forename: Malcolm
authors.initials: M
authors.lastname: Macleod
authors.affiliation: SciOme LLC, Research Triangle Park, 2 Davis Drive, 27709, NC, USA.
authors.forename: Ruchir R
authors.initials: RR
authors.lastname: Shah
authors.affiliation: Division of the National Toxicology Program, National Institute of Environmental Health Sciences, National Institutes of Health, Department of Health and Human Services, Research Triangle Park, NC, USA.
authors.forename: Kristina
authors.initials: K
authors.lastname: Thayer
mesh_terms: D000465:Algorithms; D057225:Data Mining; D016208:Databases, Factual; D016247:Information Storage and Retrieval; D016014:Linear Models; D000069550:Machine Learning; D012984:Software; D000078202:Systematic Reviews as Topic
keywords: Literature prioritization; SWIFT-Review; Scoping reports; Software; Systematic review