title: Automated annotation of functional imaging experiments via multi-label classification.
abstract: Identifying the experimental methods in human neuroimaging papers is important for grouping meaningfully similar experiments for meta-analyses. Currently, this can only be done by human readers. We present the performance of common machine learning (text mining) methods applied to the problem of automatically classifying or labeling this literature. Labeling terms are from the Cognitive Paradigm Ontology (CogPO), the text corpora are abstracts of published functional neuroimaging papers, and the methods use the performance of a human expert as training data. We aim to replicate the expert's annotation of multiple labels per abstract identifying the experimental stimuli, cognitive paradigms, response types, and other relevant dimensions of the experiments. We use several standard machine learning methods: naive Bayes (NB), k-nearest neighbor, and support vector machines (specifically SMO or sequential minimal optimization). Exact match performance ranged from only 15% in the worst cases to 78% in the best cases. NB methods combined with binary relevance transformations performed strongly and were robust to overfitting. This collection of results demonstrates what can be achieved with off-the-shelf software components and little to no pre-processing of raw text.
authors.affiliation: Department of Computer Science, University of New Mexico Albuquerque, NM, USA ; Mind Research Network Albuquerque, NM, USA ; Conjectural Systems Atlanta, GA, USA.
authors.forename: Matthew D
authors.initials: MD
authors.lastname: Turner
authors.affiliation: Department of Computer Science, University of New Mexico Albuquerque, NM, USA.
authors.forename: Chayan
authors.initials: C
authors.lastname: Chakrabarti
authors.affiliation: Department of Computer Science, University of New Mexico Albuquerque, NM, USA.
authors.forename: Thomas B
authors.initials: TB
authors.lastname: Jones
authors.affiliation: Department of Computer Science, University of New Mexico Albuquerque, NM, USA.
authors.forename: Jiawei F
authors.initials: JF
authors.lastname: Xu
authors.affiliation: Research Imaging Center, University of Texas Health Science Center San Antonio, TX, USA.
authors.forename: Peter T
authors.initials: PT
authors.lastname: Fox
authors.affiliation: Department of Computer Science, University of New Mexico Albuquerque, NM, USA.
authors.forename: George F
authors.initials: GF
authors.lastname: Luger
authors.affiliation: Department of Physics, Florida International University Miami, FL, USA.
authors.forename: Angela R
authors.initials: AR
authors.lastname: Laird
authors.affiliation: Mind Research Network Albuquerque, NM, USA ; Conjectural Systems Atlanta, GA, USA ; Department of Psychology and Neuroscience Institute, Georgia State University Atlanta, GA, USA.
authors.forename: Jessica A
authors.initials: JA
authors.lastname: Turner
keywords: CogPO; annotations; bioinformatics; data mining; multi-label classification; neuroimaging; text mining