title: An evaluation, comparison, and accurate benchmarking of several publicly available MS/MS search algorithms: sensitivity and specificity analysis.
abstract: MS/MS and associated database search algorithms are essential proteomic tools for identifying peptides. Due to their widespread use, it is now time to perform a systematic analysis of the various algorithms currently in use. Using blood specimens used in the HUPO Plasma Proteome Project, we have evaluated five search algorithms with respect to their sensitivity and specificity, and have also accurately benchmarked them based on specified false-positive (FP) rates. Spectrum Mill and SEQUEST performed well in terms of sensitivity, but were inferior to MASCOT, X!Tandem, and Sonar in terms of specificity. Overall, MASCOT, a probabilistic search algorithm, correctly identified most peptides based on a specified FP rate. The rescoring algorithm, PeptideProphet, enhanced the overall performance of the SEQUEST algorithm, as well as provided predictable FP error rates. Ideally, score thresholds should be calculated for each peptide spectrum or minimally, derived from a reversed-sequence search as demonstrated in this study based on a validated data set. The availability of open-source search algorithms, such as X!Tandem, makes it feasible to further improve the validation process (manual or automatic) on the basis of "consensus scoring", i.e., the use of multiple (at least two) search algorithms to reduce the number of FPs. complement.
authors.affiliation: Joint ProteomicS Laboratory, Ludwig Institute for Cancer Research (Melbourne Branch)/Walter and Eliza Hall Institute of Medical Research, Parkville, Victoria, Australia.
authors.forename: Eugene A
authors.initials: EA
authors.lastname: Kapp
authors.forename: Frédéric
authors.initials: F
authors.lastname: Schütz
authors.forename: Lisa M
authors.initials: LM
authors.lastname: Connolly
authors.forename: John A
authors.initials: JA
authors.lastname: Chakel
authors.forename: Jose E
authors.initials: JE
authors.lastname: Meza
authors.forename: Christine A
authors.initials: CA
authors.lastname: Miller
authors.forename: David
authors.initials: D
authors.lastname: Fenyo
authors.forename: Jimmy K
authors.initials: JK
authors.lastname: Eng
authors.forename: Joshua N
authors.initials: JN
authors.lastname: Adkins
authors.forename: Gilbert S
authors.initials: GS
authors.lastname: Omenn
authors.forename: Richard J
authors.initials: RJ
authors.lastname: Simpson
mesh_terms: D000465:Algorithms; D019985:Benchmarking; D001798:Blood Proteins / Q000737:chemistry*; D019295:Computational Biology; D030562:Databases, Protein; D005189:False Positive Reactions; D006801:Humans; D020407:Internet; D013058:Mass Spectrometry / Q000379:methods*; D010455:Peptides / Q000737:chemistry; D040901:Proteomics / Q000379:methods*; D012372:ROC Curve; D012015:Reference Standards; D012680:Sensitivity and Specificity; D012984:Software; D014357:Trypsin / Q000494:pharmacology
chemical_list: D001798:Blood Proteins; D010455:Peptides; D014357:Trypsin