title: GENETAG: a tagged corpus for gene/protein named entity recognition.
abstract: BACKGROUND
  Named entity recognition (NER) is an important first step for text mining the biomedical literature. Evaluating the performance of biomedical NER systems is impossible without a standardized test corpus. The annotation of such a corpus for gene/protein name NER is a difficult process due to the complexity of gene/protein names. We describe the construction and annotation of GENETAG, a corpus of 20K MEDLINE sentences for gene/protein NER. 15K GENETAG sentences were used for the BioCreAtIvE Task 1A Competition.
  
  
  RESULTS
  To ensure heterogeneity of the corpus, MEDLINE sentences were first scored for term similarity to documents with known gene names, and 10K high- and 10K low-scoring sentences were chosen at random. The original 20K sentences were run through a gene/protein name tagger, and the results were modified manually to reflect a wide definition of gene/protein names subject to a specificity constraint, a rule that required the tagged entities to refer to specific entities. Each sentence in GENETAG was annotated with acceptable alternatives to the gene/protein names it contained, allowing for partial matching with semantic constraints. Semantic constraints are rules requiring the tagged entity to contain its true meaning in the sentence context. Application of these constraints results in a more meaningful measure of the performance of an NER system than unrestricted partial matching.
  
  
  CONCLUSIONS
  The annotation of GENETAG required intricate manual judgments by annotators which hindered tagging consistency. The data were pre-segmented into words, to provide indices supporting comparison of system responses to the "gold standard". However, character-based indices would have been more robust than word-based indices. GENETAG Train, Test and Round1 data and ancillary programs are freely available at ftp://ftp.ncbi.nlm.nih.gov/pub/tanabe/GENETAG.tar.gz. A newer version of GENETAG-05, will be released later this year.
authors.affiliation: National Center for Biotechnology Information, National Library of Medicine, NIH, 8600 Rockville Pike, Bethesda, MD 20894, USA. tanabe@ncbi.nlm.nih.gov
authors.forename: Lorraine
authors.initials: L
authors.lastname: Tanabe
authors.forename: Natalie
authors.initials: N
authors.lastname: Xie
authors.forename: Lynne H
authors.initials: LH
authors.lastname: Thom
authors.forename: Wayne
authors.initials: W
authors.lastname: Matten
authors.forename: W John
authors.initials: WJ
authors.lastname: Wilbur
mesh_terms: D000818:Animals; D005796:Genes; D006801:Humans; D016239:MEDLINE; D011506:Proteins / Q000145:classification* / Q000235:genetics*; D021641:Recognition, Psychology; D009626:Terminology as Topic
chemical_list: D011506:Proteins