title: Benchmarking of the 2010 BioCreative Challenge III text-mining competition by the BioGRID and MINT interaction databases.
abstract: BACKGROUND
  The vast amount of data published in the primary biomedical literature represents a challenge for the automated extraction and codification of individual data elements. Biological databases that rely solely on manual extraction by expert curators are unable to comprehensively annotate the information dispersed across the entire biomedical literature. The development of efficient tools based on natural language processing (NLP) systems is essential for the selection of relevant publications, identification of data attributes and partially automated annotation. One of the tasks of the Biocreative 2010 Challenge III was devoted to the evaluation of NLP systems developed to identify articles for curation and extraction of protein-protein interaction (PPI) data.
  
  
  RESULTS
  The Biocreative 2010 competition addressed three tasks: gene normalization, article classification and interaction method identification. The BioGRID and MINT protein interaction databases both participated in the generation of the test publication set for gene normalization, annotated the development and test sets for article classification, and curated the test set for interaction method classification. These test datasets served as a gold standard for the evaluation of data extraction algorithms.
  
  
  CONCLUSIONS
  The development of efficient tools for extraction of PPI data is a necessary step to achieve full curation of the biomedical literature. NLP systems can in the first instance facilitate expert curation by refining the list of candidate publications that contain PPI data; more ambitiously, NLP approaches may be able to directly extract relevant information from full-text articles for rapid inspection by expert curators. Close collaboration between biological databases and NLP systems developers will continue to facilitate the long-term objectives of both disciplines.
authors.affiliation: School of Biological Sciences, University of Edinburgh, Edinburgh EH9 3JR, UK. a.aryamontri@ed.ac.uk
authors.forename: Andrew
authors.initials: A
authors.lastname: Chatr-Aryamontri
authors.forename: Andrew
authors.initials: A
authors.lastname: Winter
authors.forename: Livia
authors.initials: L
authors.lastname: Perfetto
authors.forename: Leonardo
authors.initials: L
authors.lastname: Briganti
authors.forename: Luana
authors.initials: L
authors.lastname: Licata
authors.forename: Marta
authors.initials: M
authors.lastname: Iannuccelli
authors.forename: Luisa
authors.initials: L
authors.lastname: Castagnoli
authors.forename: Gianni
authors.initials: G
authors.lastname: Cesareni
authors.forename: Mike
authors.initials: M
authors.lastname: Tyers
mesh_terms: D000465:Algorithms; D057225:Data Mining / Q000592:standards; D030562:Databases, Protein; D005796:Genes; D006801:Humans; D009323:Natural Language Processing