title: Statistical evaluation of pairwise protein sequence comparison with the Bayesian bootstrap.
abstract: BACKGROUND
  Protein sequence comparison methods are routinely used to infer the intricate network of evolutionary relationships found within the rapidly growing library of protein sequences, and thereby to predict the structure and function of uncharacterized proteins. In the present study, we detail an improved statistical benchmark of pairwise protein sequence comparison algorithms. We use bootstrap resampling techniques to determine standard statistical errors and to estimate the confidence of our conclusions. We show that the underlying structure within benchmark databases causes Efron's standard, non-parametric bootstrap to be biased. Consequently, the standard bootstrap underpredicts average performance when used in the context of evaluating sequence comparison methods. We have developed, as an alternative, an unbiased statistical evaluation based on the Bayesian bootstrap, a resampling method operationally similar to the standard bootstrap.
  
  
  RESULTS
  We apply our analysis to the comparative study of amino acid substitution matrix families and find that using modern matrices results in a small, but statistically significant improvement in remote homology detection compared with the classic PAM and BLOSUM matrices.
  
  
  BACKGROUND
  The sequence sets and code for performing these analyses are available from http://compbio.berkeley.edu/.
  
  
  BACKGROUND
  brenner@compbio.berkeley.edu.
authors.affiliation: Department of Bioengineering, University of California, Berkeley, 94720, USA.
authors.forename: Gavin A
authors.initials: GA
authors.lastname: Price
authors.forename: Gavin E
authors.initials: GE
authors.lastname: Crooks
authors.forename: Richard E
authors.initials: RE
authors.lastname: Green
authors.forename: Steven E
authors.initials: SE
authors.lastname: Brenner
mesh_terms: D000465:Algorithms; D000595:Amino Acid Sequence; D001499:Bayes Theorem; D003627:Data Interpretation, Statistical; D008958:Models, Molecular; D015233:Models, Statistical; D008969:Molecular Sequence Data; D011506:Proteins / Q000032:analysis / Q000737:chemistry*; D016415:Sequence Alignment / Q000379:methods*; D020539:Sequence Analysis, Protein / Q000379:methods*
chemical_list: D011506:Proteins