Automating document classification for the Immune Epitope Database.
BACKGROUND
The Immune Epitope Database contains information on immune epitopes curated manually from the scientific literature. Like similar projects in other knowledge domains, significant effort is spent on identifying which articles are relevant for this purpose.


RESULTS
We here report our experience in automating this process using Naïve Bayes classifiers trained on 20,910 abstracts classified by domain experts. Improvements on the basic classifier performance were made by a) utilizing information stored in PubMed beyond the abstract itself b) applying standard feature selection criteria and c) extracting domain specific feature patterns that e.g. identify peptides sequences. We have implemented the classifier into the curation process determining if abstracts are clearly relevant, clearly irrelevant, or if no certain classification can be made, in which case the abstracts are manually classified. Testing this classification scheme on an independent dataset, we achieve 95% sensitivity and specificity in the 51.1% of abstracts that were automatically classified.


CONCLUSIONS
By implementing text classification, we have sped up the reference selection process without sacrificing sensitivity or specificity of the human expert classification. This study provides both practical recommendations for users of text classification tools, as well as a large dataset which can serve as a benchmark for tool developers.
Wang
Peng
P
The La Jolla Institute for Allergy and Immunology, La Jolla, CA 92037, USA. pwang@liai.org <pwang@liai.org>
Morgan
Alexander A
AA
Zhang
Qing
Q
Sette
Alessandro
A
Peters
Bjoern
B
D000043:Abstracting and Indexing / Q000379:methods; D001185:Artificial Intelligence; D016208:Databases, Factual; D004282:Documentation / Q000379:methods; D018604:Epitope Mapping / Q000379:methods*; D000939:Epitopes / Q000737:chemistry* / Q000145:classification / Q000276:immunology*; D016247:Information Storage and Retrieval / Q000379:methods; D009323:Natural Language Processing; D010506:Periodicals as Topic; D039781:PubMed
D000939:Epitopes