An evaluation, comparison, and accurate benchmarking of several publicly available MS/MS search algorithms: sensitivity and specificity analysis.
MS/MS and associated database search algorithms are essential proteomic tools for identifying peptides. Due to their widespread use, it is now time to perform a systematic analysis of the various algorithms currently in use. Using blood specimens used in the HUPO Plasma Proteome Project, we have evaluated five search algorithms with respect to their sensitivity and specificity, and have also accurately benchmarked them based on specified false-positive (FP) rates. Spectrum Mill and SEQUEST performed well in terms of sensitivity, but were inferior to MASCOT, X!Tandem, and Sonar in terms of specificity. Overall, MASCOT, a probabilistic search algorithm, correctly identified most peptides based on a specified FP rate. The rescoring algorithm, PeptideProphet, enhanced the overall performance of the SEQUEST algorithm, as well as provided predictable FP error rates. Ideally, score thresholds should be calculated for each peptide spectrum or minimally, derived from a reversed-sequence search as demonstrated in this study based on a validated data set. The availability of open-source search algorithms, such as X!Tandem, makes it feasible to further improve the validation process (manual or automatic) on the basis of "consensus scoring", i.e., the use of multiple (at least two) search algorithms to reduce the number of FPs. complement.
Kapp
Eugene A
EA
Joint ProteomicS Laboratory, Ludwig Institute for Cancer Research (Melbourne Branch)/Walter and Eliza Hall Institute of Medical Research, Parkville, Victoria, Australia.
Schütz
Frédéric
F
Connolly
Lisa M
LM
Chakel
John A
JA
Meza
Jose E
JE
Miller
Christine A
CA
Fenyo
David
D
Eng
Jimmy K
JK
Adkins
Joshua N
JN
Omenn
Gilbert S
GS
Simpson
Richard J
RJ
D000465:Algorithms; D019985:Benchmarking; D001798:Blood Proteins / Q000737:chemistry*; D019295:Computational Biology; D030562:Databases, Protein; D005189:False Positive Reactions; D006801:Humans; D020407:Internet; D013058:Mass Spectrometry / Q000379:methods*; D010455:Peptides / Q000737:chemistry; D040901:Proteomics / Q000379:methods*; D012372:ROC Curve; D012015:Reference Standards; D012680:Sensitivity and Specificity; D012984:Software; D014357:Trypsin / Q000494:pharmacology
D001798:Blood Proteins; D010455:Peptides; D014357:Trypsin