title: CALBC silver standard corpus.
abstract: The CALBC initiative aims to provide a large-scale biomedical text corpus that contains semantic annotations for named entities of different kinds. The generation of this corpus requires that the annotations from different automatic annotation systems be harmonized. In the first phase, the annotation systems from five participants (EMBL-EBI, EMC Rotterdam, NLM, JULIE Lab Jena, and Linguamatics) were gathered. All annotations were delivered in a common annotation format that included concept identifiers in the boundary assignments and that enabled comparison and alignment of the results. During the harmonization phase, the results produced from those different systems were integrated in a single harmonized corpus ("silver standard" corpus) by applying a voting scheme. We give an overview of the processed data and the principles of harmonization--formal boundary reconciliation and semantic matching of named entities. Finally, all submissions of the participants were evaluated against that silver standard corpus. We found that species and disease annotations are better standardized amongst the partners than the annotations of genes and proteins. The raw corpus is now available for additional named entity annotations. Parts of it will be made available later on for a public challenge. We expect that we can improve corpus building activities both in terms of the numbers of named entity classes being covered, as well as the size of the corpus in terms of annotated documents.
authors.affiliation: EMBL Outstation-Hinxton, European Bioinformatics Institute, Hinxton, Cambridge CB101SD, UK. rebholz@ebi.ac.uk
authors.forename: Dietrich
authors.initials: D
authors.lastname: Rebholz-Schuhmann
authors.forename: Antonio José
authors.initials: AJ
authors.lastname: Jimeno Yepes
authors.forename: Erik M
authors.initials: EM
authors.lastname: Van Mulligen
authors.forename: Ning
authors.initials: N
authors.lastname: Kang
authors.forename: Jan
authors.initials: J
authors.lastname: Kors
authors.forename: David
authors.initials: D
authors.lastname: Milward
authors.forename: Peter
authors.initials: P
authors.lastname: Corbett
authors.forename: Ekaterina
authors.initials: E
authors.lastname: Buyko
authors.forename: Elena
authors.initials: E
authors.lastname: Beisswanger
authors.forename: Udo
authors.initials: U
authors.lastname: Hahn
mesh_terms: D019295:Computational Biology / Q000592:standards*; D003299:Cooperative Behavior; D057225:Data Mining / Q000592:standards* / Q000706:statistics & numerical data; D016208:Databases, Factual / Q000706:statistics & numerical data; D017432:Unified Medical Language System