title: Monitoring named entity recognition: the League Table.
abstract: BACKGROUND
  Named entity recognition (NER) is an essential step in automatic text processing pipelines. A number of solutions have been presented and evaluated against gold standard corpora (GSC). The benchmarking against GSCs is crucial, but left to the individual researcher. Herewith we present a League Table web site, which benchmarks NER solutions against selected public GSCs, maintains a ranked list and archives the annotated corpus for future comparisons.
  
  
  RESULTS
  The web site enables access to the different GSCs in a standardized format (IeXML). Upon submission of the annotated corpus the user has to describe the specification of the used solution and then uploads the annotated corpus for evaluation. The performance of the system is measured against one or more GSCs and the results are then added to the web site ("League Table"). It displays currently the results from publicly available NER solutions from the Whatizit infrastructure for future comparisons.
  
  
  CONCLUSIONS
  The League Table enables the evaluation of NER solutions in a standardized infrastructure and monitors the results long-term. For access please go to http://wwwdev.ebi.ac.uk/Rebholz-srv/calbc/assessmentGSC/.
  
  
  BACKGROUND
  rebholz@ifi.uzh.ch.
authors.affiliation: Department of Computational Linguistics, University of Zurich, Zürich, Switzerland. rebholz@ifi.uzh.ch.
authors.forename: Dietrich
authors.initials: D
authors.lastname: Rebholz-Schuhmann
authors.forename: Senay
authors.initials: S
authors.lastname: Kafkas
authors.forename: Jee-Hyub
authors.initials: JH
authors.lastname: Kim
authors.forename: Antonio
authors.initials: A
authors.lastname: Jimeno Yepes
authors.forename: Ian
authors.initials: I
authors.lastname: Lewin
references.citation: Int J Med Inform. 2006 Jun;75(6):496-500
references.pmid: 16085453
references.citation: Bioinformatics. 2009 Aug 1;25(15):1997-8
references.pmid: 19414535
references.citation: Genome Biol. 2008;9 Suppl 2:S2
references.pmid: 18834493
references.citation: J Biomed Semantics. 2013 Oct 11;4(1):28
references.pmid: 24112383
references.citation: BMC Bioinformatics. 2005;6 Suppl 1:S3
references.pmid: 15960837
references.citation: Proteins. 2011;79 Suppl 10:1-5
references.pmid: 21997831
references.citation: BMC Bioinformatics. 2005;6 Suppl 1:S1
references.pmid: 15960821
references.citation: Nat Rev Genet. 2012 Dec;13(12):829-39
references.pmid: 23150036
references.citation: Bioinformatics. 2003;19 Suppl 1:i180-2
references.pmid: 12855455
references.citation: Sci Signal. 2011 Aug 30;4(189):mr7
references.pmid: 21900204
references.citation: J Biomed Semantics. 2011 Oct 06;2 Suppl 5:S11
references.pmid: 22166494