Assessing the statistical validity of proteomics based biomarkers.
A strategy is presented for the statistical validation of discrimination models in proteomics studies. Several existing tools are combined to form a solid statistical basis for biomarker discovery that should precede a biochemical validation of any biomarker. These tools consist of permutation tests, single and double cross-validation. The cross-validation steps can simply be combined with a new variable selection method, called rank products. The strategy is especially suited for the low-samples-to-variables-ratio (undersampling) case, as is often encountered in proteomics and metabolomics studies. As a classification method, principal component discriminant analysis is used; however, the methodology can be used with any classifier. A dataset containing serum samples from Gaucher patients and healthy controls serves as a test case. Double cross-validation shows that the sensitivity of the model is 89% and the specificity 90%. Potential putative biomarkers are identified using the novel variable selection method. Results from permutation tests support the choice of double cross-validation as the tool for determining error rates when the modelling procedure involves a tuneable parameter. This shows that even cross-validation does not guarantee unbiased results. The validation of discrimination models with a combination of permutation tests and double cross-validation helps to avoid erroneous results which may result from the undersampling.
Smit
Suzanne
S
Swammerdam Institute for Life Sciences, Universiteit van-Amsterdam, Nieuwe Achtergracht 166, 1018 WV Amsterdam, The Netherlands.
van Breemen
Mariëlle J
MJ
Hoefsloot
Huub C J
HC
Smilde
Age K
AK
Aerts
Johannes M F G
JM
de Koster
Chris G
CG
D000293:Adolescent; D000328:Adult; D000368:Aged; D015415:Biomarkers / Q000097:blood / Q000737:chemistry; D005260:Female; D006801:Humans; D008297:Male; D013058:Mass Spectrometry; D008875:Middle Aged; D040901:Proteomics / Q000145:classification / Q000379:methods* / Q000592:standards*; D015203:Reproducibility of Results; D013223:Statistics as Topic
D015415:Biomarkers