title: Standardized benchmarking in the quest for orthologs.
abstract: Achieving high accuracy in orthology inference is essential for many comparative, evolutionary and functional genomic analyses, yet the true evolutionary history of genes is generally unknown and orthologs are used for very different applications across phyla, requiring different precision-recall trade-offs. As a result, it is difficult to assess the performance of orthology inference methods. Here, we present a community effort to establish standards and an automated web-based service to facilitate orthology benchmarking. Using this service, we characterize 15 well-established inference methods and resources on a battery of 20 different benchmarks. Standardized benchmarking provides a way for users to identify the most effective methods for the problem at hand, sets a minimum requirement for new tools and resources, and guides the development of more accurate orthology inference methods.
authors.affiliation: Department of Computer Science, ETH Zurich, Zurich, Switzerland.|Computational Biochemistry Research Group, Swiss Institute of Bioinformatics, Zurich, Switzerland.
authors.forename: Adrian M
authors.initials: AM
authors.lastname: Altenhoff
authors.affiliation: Swiss-Prot Group, Swiss Institute of Bioinformatics, Geneva, Switzerland.
authors.forename: Brigitte
authors.initials: B
authors.lastname: Boeckmann
authors.affiliation: Bioinformatics and Genomics Programme, Centre for Genomic Regulation, Barcelona Institute of Science and Technology, Barcelona, Spain.|Universitat Pompeu Fabra, Barcelona, Spain.|Yeast and Basidiomycete Research Group, CBS Fungal Biodiversity Centre, Utrecht, the Netherlands.
authors.forename: Salvador
authors.initials: S
authors.lastname: Capella-Gutierrez
authors.affiliation: Department of Genetics, Evolution, and Environment, University College London, London, UK.
authors.forename: Daniel A
authors.initials: DA
authors.lastname: Dalquen
authors.affiliation: Center for Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA.
authors.forename: Todd
authors.initials: T
authors.lastname: DeLuca
authors.affiliation: Structural and Computational Biology Unit, European Molecular Biology Laboratory, Heidelberg, Germany.
authors.forename: Kristoffer
authors.identifier: 0000-0003-4285-6993
authors.initials: K
authors.lastname: Forslund
authors.affiliation: Structural and Computational Biology Unit, European Molecular Biology Laboratory, Heidelberg, Germany.
authors.forename: Jaime
authors.initials: J
authors.lastname: Huerta-Cepas
authors.affiliation: Department of Life Sciences, Natural History Museum, London, UK.
authors.forename: Benjamin
authors.initials: B
authors.lastname: Linard
authors.affiliation: Université Paris-Sud, Laboratoire de Recherche en Informatique, Orsay, France.|Université Paris-Sud, Institute for Integrative Biology of the Cell, Orsay, France.
authors.forename: Cécile
authors.initials: C
authors.lastname: Pereira
authors.affiliation: Bioinformatics and Genomics Programme, Centre for Genomic Regulation, Barcelona Institute of Science and Technology, Barcelona, Spain.
authors.forename: Leszek P
authors.initials: LP
authors.lastname: Pryszcz
authors.affiliation: European Molecular Biology Laboratory, European Bioinformatics Institute, Wellcome Trust Genome Campus, Hinxton, UK.
authors.forename: Fabian
authors.initials: F
authors.lastname: Schreiber
authors.affiliation: European Molecular Biology Laboratory, European Bioinformatics Institute, Wellcome Trust Genome Campus, Hinxton, UK.
authors.forename: Alan Sousa
authors.initials: AS
authors.lastname: da Silva
authors.affiliation: Institute of Molecular Life Sciences, University of Zurich, Zurich, Switzerland.|Bioinformatics Systems Biology Group, Swiss Institute of Bioinformatics, Zurich, Switzerland.
authors.forename: Damian
authors.initials: D
authors.lastname: Szklarczyk
authors.affiliation: Department of Computer Science, ETH Zurich, Zurich, Switzerland.
authors.forename: Clément-Marie
authors.initials: CM
authors.lastname: Train
authors.affiliation: Structural and Computational Biology Unit, European Molecular Biology Laboratory, Heidelberg, Germany.|Germany Molecular Medicine Partnership Unit, University Hospital Heidelberg and European Molecular Biology Laboratory, Heidelberg, Germany.|Max Delbrück Centre for Molecular Medicine, Berlin, Germany.
authors.forename: Peer
authors.initials: P
authors.lastname: Bork
authors.affiliation: LBGI, Computer Science Department, ICube, University of Strasbourg, Strasbourg, France.
authors.forename: Odile
authors.initials: O
authors.lastname: Lecompte
authors.affiliation: Institute of Molecular Life Sciences, University of Zurich, Zurich, Switzerland.|Bioinformatics Systems Biology Group, Swiss Institute of Bioinformatics, Zurich, Switzerland.
authors.forename: Christian
authors.identifier: 0000-0001-7734-9102
authors.initials: C
authors.lastname: von Mering
authors.affiliation: Swiss-Prot Group, Swiss Institute of Bioinformatics, Geneva, Switzerland.|Vital-IT, Swiss Institute of Bioinformatics, Lausanne, Switzerland.|Center for Integrative Genomics, University of Lausanne, Lausanne, Switzerland.
authors.forename: Ioannis
authors.identifier: 0000-0002-3413-6841
authors.initials: I
authors.lastname: Xenarios
authors.affiliation: Department of Bioengineering, University of California, Berkeley, California, USA.
authors.forename: Kimmen
authors.initials: K
authors.lastname: Sjölander
authors.affiliation: The Novo Nordisk Foundation Center for Protein Research, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark.
authors.forename: Lars Juhl
authors.initials: LJ
authors.lastname: Jensen
authors.affiliation: European Molecular Biology Laboratory, European Bioinformatics Institute, Wellcome Trust Genome Campus, Hinxton, UK.
authors.forename: Maria J
authors.initials: MJ
authors.lastname: Martin
authors.affiliation: European Molecular Biology Laboratory, European Bioinformatics Institute, Wellcome Trust Genome Campus, Hinxton, UK.
authors.forename: Matthieu
authors.initials: M
authors.lastname: Muffato
authors.affiliation: Bioinformatics and Genomics Programme, Centre for Genomic Regulation, Barcelona Institute of Science and Technology, Barcelona, Spain.|Universitat Pompeu Fabra, Barcelona, Spain.|Institució Catalana de Recerca I Estudis Avançats, Barcelona, Spain.
authors.forename: Toni
authors.initials: T
authors.lastname: Gabaldón
authors.affiliation: Genomics Division, Lawrence Berkeley National Laboratory, Berkeley, California, USA.
authors.forename: Suzanna E
authors.initials: SE
authors.lastname: Lewis
authors.affiliation: Division of Bioinformatics, Department of Preventive Medicine, University of Southern California, Los Angeles, California, USA.
authors.forename: Paul D
authors.initials: PD
authors.lastname: Thomas
authors.affiliation: Stockholm Bioinformatics Center, Department of Biochemistry and Biophysics, Stockholm University, Science for Life Laboratory, Solna, Sweden.
authors.forename: Erik
authors.initials: E
authors.lastname: Sonnhammer
authors.affiliation: Department of Genetics, Evolution, and Environment, University College London, London, UK.|Center for Integrative Genomics, University of Lausanne, Lausanne, Switzerland.|Department of Ecology and Evolution, University of Lausanne, Lausanne, Switzerland.|Department of Computer Science, University College London, London, UK.|Swiss Institute of Bioinformatics, Biophore Building, Lausanne, Switzerland.
authors.forename: Christophe
authors.identifier: 0000-0002-2170-853X
authors.initials: C
authors.lastname: Dessimoz
mesh_terms: D001105:Archaea / Q000145:classification / Q000235:genetics; D001419:Bacteria / Q000145:classification / Q000235:genetics; D019295:Computational Biology / Q000379:methods / Q000592:standards*; D030541:Databases, Genetic; D056890:Eukaryota / Q000145:classification / Q000235:genetics; D063990:Gene Ontology; D023281:Genomics / Q000379:methods / Q000592:standards*; D008957:Models, Genetic; D010802:Phylogeny; D040901:Proteomics / Q000379:methods / Q000592:standards*; D020539:Sequence Analysis, Protein; D017385:Sequence Homology; D013045:Species Specificity