title: The BRaliBase dent-a tale of benchmark design and interpretation.
abstract: BRaliBase is a widely used benchmark for assessing the accuracy of RNA secondary structure alignment methods. In most case studies based on the BRaliBase benchmark, one can observe a puzzling drop in accuracy in the 40-60% sequence identity range, the so-called 'BRaliBase Dent'. In this article, we show this dent is owing to a bias in the composition of the BRaliBase benchmark, namely the inclusion of a disproportionate number of transfer RNAs, which exhibit a conserved secondary structure. Our analysis, aside of its interest regarding the specific case of the BRaliBase benchmark, also raises important questions regarding the design and use of benchmarks in computational biology.
authors.affiliation: Division of Cardiology, University of Nebraska Medical Center, USA.
authors.forename: Benedikt
authors.initials: B
authors.lastname: Löwes
authors.affiliation: Department of Mathematics, Simon Fraser University, Burnaby, BC, Canada.
authors.forename: Cedric
authors.initials: C
authors.lastname: Chauve
authors.affiliation: LIX, CNRS/Inria AMIB, Ecole Polytechnique, Palaiseau, France.
authors.forename: Yann
authors.initials: Y
authors.lastname: Ponty
authors.affiliation: Institute for Bioinformatics, Bielefeld University, Bielefeld, Germany.
authors.forename: Robert
authors.initials: R
authors.lastname: Giegerich
mesh_terms: D000465:Algorithms; D019985:Benchmarking; D019295:Computational Biology; D009690:Nucleic Acid Conformation; D016415:Sequence Alignment; D017423:Sequence Analysis, RNA; D012984:Software
keywords: RNA family database; RNA structural alignment; benchmark