title: A realistic assessment of methods for extracting gene/protein interactions from free text.
abstract: BACKGROUND
  The automated extraction of gene and/or protein interactions from the literature is one of the most important targets of biomedical text mining research. In this paper we present a realistic evaluation of gene/protein interaction mining relevant to potential non-specialist users. Hence we have specifically avoided methods that are complex to install or require reimplementation, and we coupled our chosen extraction methods with a state-of-the-art biomedical named entity tagger.
  
  
  RESULTS
  Our results show: that performance across different evaluation corpora is extremely variable; that the use of tagged (as opposed to gold standard) gene and protein names has a significant impact on performance, with a drop in F-score of over 20 percentage points being commonplace; and that a simple keyword-based benchmark algorithm when coupled with a named entity tagger outperforms two of the tools most widely used to extract gene/protein interactions.
  
  
  CONCLUSIONS
  In terms of availability, ease of use and performance, the potential non-specialist user community interested in automatically extracting gene and/or protein interactions from free text is poorly served by current tools and systems. The public release of extraction tools that are easy to install and use, and that achieve state-of-art levels of performance should be treated as a high priority by the biomedical text mining community.
authors.affiliation: School of Crystallography and Institute of Structural and Molecular Biology, Birkbeck College, University of London, Malet Street, London WC1E 7HX UK. r.kabiljo@mail.cryst.bbk.ac.uk
authors.forename: Renata
authors.initials: R
authors.lastname: Kabiljo
authors.forename: Andrew B
authors.initials: AB
authors.lastname: Clegg
authors.forename: Adrian J
authors.initials: AJ
authors.lastname: Shepherd
mesh_terms: D019295:Computational Biology / Q000379:methods*; D030541:Databases, Genetic; D030562:Databases, Protein; D005796:Genes; D025941:Protein Interaction Mapping / Q000379:methods; D011506:Proteins / Q000737:chemistry* / Q000378:metabolism
chemical_list: D011506:Proteins