title: A new algorithm for reducing the workload of experts in performing systematic reviews.
abstract: OBJECTIVE
  To determine whether a factorized version of the complement naïve Bayes (FCNB) classifier can reduce the time spent by experts reviewing journal articles for inclusion in systematic reviews of drug class efficacy for disease treatment.
  
  
  METHODS
  The proposed classifier was evaluated on a test collection built from 15 systematic drug class reviews used in previous work. The FCNB classifier was constructed to classify each article as containing high-quality, drug class-specific evidence or not. Weight engineering (WE) techniques were added to reduce underestimation for Medical Subject Headings (MeSH)-based and Publication Type (PubType)-based features. Cross-validation experiments were performed to evaluate the classifier's parameters and performance.
  
  
  METHODS
  Work saved over sampling (WSS) at no less than a 95% recall was used as the main measure of performance.
  
  
  RESULTS
  The minimum workload reduction for a systematic review for one topic, achieved with a FCNB/WE classifier, was 8.5%; the maximum was 62.2% and the average over the 15 topics was 33.5%. This is 15.0% higher than the average workload reduction obtained using a voting perceptron-based automated citation classification system.
  
  
  CONCLUSIONS
  The FCNB/WE classifier is simple, easy to implement, and produces significantly better results in reducing the workload than previously achieved. The results support it being a useful algorithm for machine-learning-based automation of systematic reviews of drug class efficacy for disease treatment.
authors.affiliation: School of Information Technology and Engineering, University of Ottawa, Ottawa, Ontario, Canada.
authors.forename: Stan
authors.initials: S
authors.lastname: Matwin
authors.forename: Alexandre
authors.initials: A
authors.lastname: Kouznetsov
authors.forename: Diana
authors.initials: D
authors.lastname: Inkpen
authors.forename: Oana
authors.initials: O
authors.lastname: Frunza
authors.forename: Peter
authors.initials: P
authors.lastname: O'Blenis
mesh_terms: D000465:Algorithms; D001331:Automation; D001499:Bayes Theorem; D003661:Decision Support Techniques; D004358:Drug Therapy; D004526:Efficiency; D019317:Evidence-Based Medicine / Q000145:classification*; D006801:Humans; D016247:Information Storage and Retrieval / Q000145:classification*; D012196:Review Literature as Topic