title: A phylogeny-based benchmarking test for orthology inference reveals the limitations of function-based validation.
abstract: Accurate orthology prediction is crucial for many applications in the post-genomic era. The lack of broadly accepted benchmark tests precludes a comprehensive analysis of orthology inference. So far, functional annotation between orthologs serves as a performance proxy. However, this violates the fundamental principle of orthology as an evolutionary definition, while it is often not applicable due to limited experimental evidence for most species. Therefore, we constructed high quality "gold standard" orthologous groups that can serve as a benchmark set for orthology inference in bacterial species. Herein, we used this dataset to demonstrate 1) why a manually curated, phylogeny-based dataset is more appropriate for benchmarking orthology than other popular practices and 2) how it guides database design and parameterization through careful error quantification. More specifically, we illustrate how function-based tests often fail to identify false assignments, misjudging the true performance of orthology inference methods. We also examined how our dataset can instruct the selection of a "core" species repertoire to improve detection accuracy. We conclude that including more genomes at the proper evolutionary distances can influence the overall quality of orthology detection. The curated gene families, called Reference Orthologous Groups, are publicly available at http://eggnog.embl.de/orthobench2.
authors.affiliation: Institute for Systems Biology, Seattle, WA, United States of America.
authors.forename: Kalliopi
authors.initials: K
authors.lastname: Trachana
authors.affiliation: Structural and Computational Biology Unit, European Molecular Biology Laboratory, Heidelberg, Germany.
authors.forename: Kristoffer
authors.initials: K
authors.lastname: Forslund
authors.affiliation: Structural and Computational Biology Unit, European Molecular Biology Laboratory, Heidelberg, Germany; Developmental Biology Unit, European Molecular Biology Laboratory, Heidelberg, Germany.
authors.forename: Tomas
authors.initials: T
authors.lastname: Larsson
authors.affiliation: Structural and Computational Biology Unit, European Molecular Biology Laboratory, Heidelberg, Germany.
authors.forename: Sean
authors.initials: S
authors.lastname: Powell
authors.affiliation: Structural and Computational Biology Unit, European Molecular Biology Laboratory, Heidelberg, Germany.
authors.forename: Tobias
authors.initials: T
authors.lastname: Doerks
authors.affiliation: Institute of Molecular Life Sciences, University of Zurich and Swiss Institute of Bioinformatics, Zurich, Switzerland.
authors.forename: Christian
authors.initials: C
authors.lastname: von Mering
authors.affiliation: Structural and Computational Biology Unit, European Molecular Biology Laboratory, Heidelberg, Germany; Max-Delbruck-Centre for Molecular Medicine, Berlin, Germany.
authors.forename: Peer
authors.initials: P
authors.lastname: Bork
mesh_terms: D001419:Bacteria / Q000145:classification; D019295:Computational Biology / Q000592:standards; D023281:Genomics; D020407:Internet; D010802:Phylogeny; D014584:User-Computer Interface