title: Comparison of automated and human assignment of MeSH terms on publicly-available molecular datasets.
abstract: Publicly available molecular datasets can be used for independent verification or investigative repurposing, but depends on the presence, consistency and quality of descriptive annotations. Annotation and indexing of molecular datasets using well-defined controlled vocabularies or ontologies enables accurate and systematic data discovery, yet the majority of molecular datasets available through public data repositories lack such annotations. A number of automated annotation methods have been developed; however few systematic evaluations of the quality of annotations supplied by application of these methods have been performed using annotations from standing public data repositories. Here, we compared manually-assigned Medical Subject Heading (MeSH) annotations associated with experiments by data submitters in the PRoteomics IDEntification (PRIDE) proteomics data repository to automated MeSH annotations derived through the National Center for Biomedical Ontology Annotator and National Library of Medicine MetaMap programs. These programs were applied to free-text annotations for experiments in PRIDE. As many submitted datasets were referenced in publications, we used the manually curated MeSH annotations of those linked publications in MEDLINE as "gold standard". Annotator and MetaMap exhibited recall performance 3-fold greater than that of the manual annotations. We connected PRIDE experiments in a network topology according to shared MeSH annotations and found 373 distinct clusters, many of which were found to be biologically coherent by network analysis. The results of this study suggest that both Annotator and MetaMap are capable of annotating public molecular datasets with a quality comparable, and often exceeding, that of the actual data submitters, highlighting a continuous need to improve and apply automated methods to molecular datasets in public data repositories to maximize their value and utility.
authors.affiliation: Division of Systems Medicine, Department of Pediatrics, Stanford, CA, 94305, USA.
authors.forename: David
authors.initials: D
authors.lastname: Ruau
authors.affiliation: School of Allied Medical Professions, The Ohio State University College of Medicine, Columbus, OH 43210, USA.
authors.forename: Michael
authors.initials: M
authors.lastname: Mbagwu
authors.affiliation: Division of Systems Medicine, Department of Pediatrics, Stanford, CA, 94305, USA; Program in Biomedical Informatics, Stanford University School of Medicine, Stanford, CA 94305, USA.
authors.forename: Joel T
authors.initials: JT
authors.lastname: Dudley
authors.affiliation: Department of Computer Science, Stanford University School of Medicine, Stanford, CA 94305, USA.
authors.forename: Vijay
authors.initials: V
authors.lastname: Krishnan
authors.affiliation: Division of Systems Medicine, Department of Pediatrics, Stanford, CA, 94305, USA. Electronic address: abutte@stanford.edu.
authors.forename: Atul J
authors.initials: AJ
authors.lastname: Butte
mesh_terms: D016208:Databases, Factual; D006801:Humans; D016247:Information Storage and Retrieval / Q000379:methods*; D046650:Medical Subject Headings; D009323:Natural Language Processing; D040901:Proteomics; D014481:United States; D018875:Vocabulary, Controlled