A new algorithm for reducing the workload of experts in performing systematic reviews.
OBJECTIVE
To determine whether a factorized version of the complement naïve Bayes (FCNB) classifier can reduce the time spent by experts reviewing journal articles for inclusion in systematic reviews of drug class efficacy for disease treatment.


METHODS
The proposed classifier was evaluated on a test collection built from 15 systematic drug class reviews used in previous work. The FCNB classifier was constructed to classify each article as containing high-quality, drug class-specific evidence or not. Weight engineering (WE) techniques were added to reduce underestimation for Medical Subject Headings (MeSH)-based and Publication Type (PubType)-based features. Cross-validation experiments were performed to evaluate the classifier's parameters and performance.


METHODS
Work saved over sampling (WSS) at no less than a 95% recall was used as the main measure of performance.


RESULTS
The minimum workload reduction for a systematic review for one topic, achieved with a FCNB/WE classifier, was 8.5%; the maximum was 62.2% and the average over the 15 topics was 33.5%. This is 15.0% higher than the average workload reduction obtained using a voting perceptron-based automated citation classification system.


CONCLUSIONS
The FCNB/WE classifier is simple, easy to implement, and produces significantly better results in reducing the workload than previously achieved. The results support it being a useful algorithm for machine-learning-based automation of systematic reviews of drug class efficacy for disease treatment.
Matwin
Stan
S
School of Information Technology and Engineering, University of Ottawa, Ottawa, Ontario, Canada.
Kouznetsov
Alexandre
A
Inkpen
Diana
D
Frunza
Oana
O
O'Blenis
Peter
P
D000465:Algorithms; D001331:Automation; D001499:Bayes Theorem; D003661:Decision Support Techniques; D004358:Drug Therapy; D004526:Efficiency; D019317:Evidence-Based Medicine / Q000145:classification*; D006801:Humans; D016247:Information Storage and Retrieval / Q000145:classification*; D012196:Review Literature as Topic