title: An Overview of BioCreative II.5.
abstract: We present the results of the BioCreative II.5 evaluation in association with the FEBS Letters experiment, where authors created Structured Digital Abstracts to capture information about protein-protein interactions. The BioCreative II.5 challenge evaluated automatic annotations from 15 text mining teams based on a gold standard created by reconciling annotations from curators, authors, and automated systems. The tasks were to rank articles for curation based on curatable protein-protein interactions; to identify the interacting proteins (using UniProt identifiers) in the positive articles (61); and to identify interacting protein pairs. There were 595 full-text articles in the evaluation test set, including those both with and without curatable protein interactions. The principal evaluation metrics were the interpolated area under the precision/recall curve (AUC iP/R), and (balanced) F-measure. For article classification, the best AUC iP/R was 0.70; for interacting proteins, the best system achieved good macroaveraged recall (0.73) and interpolated area under the precision/recall curve (0.58), after filtering incorrect species and mapping homonymous orthologs; for interacting protein pairs, the top (filtered, mapped) recall was 0.42 and AUC iP/R was 0.29. Ensemble systems improved performance for the interacting protein task.
authors: Structural Biology and BioComputing Programme, Spanish National Cancer Research Centre, Madrid, Spain. valencia@cnio.es
authors: Florian
authors: F
authors: Leitner
authors: Scott A
authors: SA
authors: Mardis
authors: Martin
authors: M
authors: Krallinger
authors: Gianni
authors: G
authors: Cesareni
authors: Lynette A
authors: LA
authors: Hirschman
authors: Alfonso
authors: A
authors: Valencia
mesh_terms: D000043:Abstracting and Indexing; D019295:Computational Biology / Q000379:methods*; D003625:Data Collection / Q000379:methods; D057225:Data Mining / Q000379:methods*; D003628:Database Management Systems; D016208:Databases, Factual; D019451:Information Management / Q000379:methods*; D009323:Natural Language Processing; D025941:Protein Interaction Mapping / Q000145:classification*