title: Translational biomarker discovery in clinical metabolomics: an introductory tutorial.
abstract: Metabolomics is increasingly being applied towards the identification of biomarkers for disease diagnosis, prognosis and risk prediction. Unfortunately among the many published metabolomic studies focusing on biomarker discovery, there is very little consistency and relatively little rigor in how researchers select, assess or report their candidate biomarkers. In particular, few studies report any measure of sensitivity, specificity, or provide receiver operator characteristic (ROC) curves with associated confidence intervals. Even fewer studies explicitly describe or release the biomarker model used to generate their ROC curves. This is surprising given that for biomarker studies in most other biomedical fields, ROC curve analysis is generally considered the standard method for performance assessment. Because the ultimate goal of biomarker discovery is the translation of those biomarkers to clinical practice, it is clear that the metabolomics community needs to start "speaking the same language" in terms of biomarker analysis and reporting-especially if it wants to see metabolite markers being routinely used in the clinic. In this tutorial, we will first introduce the concept of ROC curves and describe their use in single biomarker analysis for clinical chemistry. This includes the construction of ROC curves, understanding the meaning of area under ROC curves (AUC) and partial AUC, as well as the calculation of confidence intervals. The second part of the tutorial focuses on biomarker analyses within the context of metabolomics. This section describes different statistical and machine learning strategies that can be used to createmulti-metabolitebiomarker models and explains how these models can be assessed using ROC curves. In the third part of the tutorial we discuss common issues and potential pitfalls associated with different analysis methods and provide readers with a list of nine recommendations for biomarker analysis and reporting. To help readers test, visualize and explore the concepts presented in this tutorial, we also introduce a web-based tool called ROCCET (ROC Curve Explorer & Tester, http://www.roccet.ca). ROCCET was originally developed as a teaching aid but it can also serve as a training and testing resource to assist metabolomics researchers build biomarker models and conduct a range of common ROC curve analyses for biomarker studies.
authors.affiliation: Department of Biological Sciences, University of Alberta, Edmonton, AB Canada.
authors.forename: Jianguo
authors.initials: J
authors.lastname: Xia
authors.forename: David I
authors.initials: DI
authors.lastname: Broadhurst
authors.forename: Michael
authors.initials: M
authors.lastname: Wilson
authors.forename: David S
authors.initials: DS
authors.lastname: Wishart
keywords: AUC; Biomarker analysis; Biomarker validation and reporting; Bootstrapping; Confidence intervals; Cross validation; Optimal threshold; ROC curve; Sample size
references.citation: Nat Genet. 2012 Jan 27;44(2):121-6
references.pmid: 22281772
references.citation: Bioinformatics. 2005 Apr 1;21(7):860-8
references.pmid: 15513990
references.citation: Clin Pharmacol Ther. 2001 Mar;69(3):89-95
references.pmid: 11240971
references.citation: Proc Natl Acad Sci U S A. 2010 May 25;107(21):9546-51
references.pmid: 20460310
references.citation: JAMA. 1990 Jan 12;263(2):275-8
references.pmid: 2403604
references.citation: IEEE/ACM Trans Comput Biol Bioinform. 2007 Apr-Jun;4(2):279-92
references.pmid: 17473320
references.citation: Nat Protoc. 2011 Jun;6(6):743-60
references.pmid: 21637195
references.citation: N Engl J Med. 2003 Jun 5;348(23):2304-12
references.pmid: 12788994
references.citation: Clin Chem. 1993 Apr;39(4):561-77
references.pmid: 8472349
references.citation: Bioinformatics. 2011 Jul 1;27(13):1878-9
references.pmid: 21551139
references.citation: Metabolomics. 2012 Jun;8(Suppl 1):146-160
references.pmid: 22593726
references.citation: Bioinformatics. 2003 Jan 22;19(2):185-93
references.pmid: 12538238
references.citation: Bioinformatics. 2007 Oct 1;23(19):2507-17
references.pmid: 17720704
references.citation: J Matern Fetal Neonatal Med. 2012 Oct;25(10):1840-7
references.pmid: 22494326
references.citation: Stat Med. 1997 Oct 15;16(19):2143-56
references.pmid: 9330425
references.citation: Anal Chem. 2006 Jul 1;78(13):4281-90
references.pmid: 16808434
references.citation: Proc Natl Acad Sci U S A. 2002 May 14;99(10):6562-6
references.pmid: 11983868
references.citation: Cancer. 1950 Jan;3(1):32-5
references.pmid: 15405679
references.citation: BMC Genomics. 2006 Jun 08;7:142
references.pmid: 16762068
references.citation: Bioanalysis. 2012 Sep;4(18):2249-64
references.pmid: 23046267
references.citation: Nucleic Acids Res. 2010 Jul;38(Web Server issue):W71-7
references.pmid: 20457745
references.citation: J Biomed Inform. 2005 Oct;38(5):404-15
references.pmid: 16198999
references.citation: Biometrics. 2003 Sep;59(3):614-23
references.pmid: 14601762
references.citation: Bioinformatics. 2010 Sep 15;26(18):2342-4
references.pmid: 20628077
references.citation: Anal Chim Acta. 2007 Jun 5;592(2):210-7
references.pmid: 17512828
references.citation: Chem Rev. 2001 Feb;101(2):445-77
references.pmid: 11712254
references.citation: J Urol. 1999 Aug;162(2):293-306
references.pmid: 10411025
references.citation: J Proteome Res. 2007 Feb;6(2):469-79
references.pmid: 17269704
references.citation: Radiology. 2003 May;227(2):309-13
references.pmid: 12732691
references.citation: Chem Soc Rev. 2011 Jan;40(1):387-426
references.pmid: 20717559
references.citation: Med Decis Making. 1989 Jul-Sep;9(3):190-5
references.pmid: 2668680
references.citation: Stat Med. 2000 May 15;19(9):1141-64
references.pmid: 10797513
references.citation: Brief Bioinform. 2012 Jan;13(1):83-97
references.pmid: 21422066
references.citation: Circulation. 2001 Apr 10;103(14):1832-7
references.pmid: 11294799
references.citation: BMC Bioinformatics. 2009 Jan 08;10:11
references.pmid: 19133141
references.citation: Clin Chem. 2004 Jul;50(7):1118-25
references.pmid: 15142978
references.citation: J Biopharm Stat. 2011 Nov;21(6):1094-9
references.pmid: 22023678
references.citation: Anal Chem. 2006 Jan 15;78(2):567-74
references.pmid: 16408941
references.citation: Bioinformatics. 2010 Apr 1;26(7):971-3
references.pmid: 20139469
references.citation: Radiology. 2004 Mar;230(3):606-12
references.pmid: 14990827
references.citation: J Natl Cancer Inst. 2001 Jul 18;93(14):1054-61
references.pmid: 11459866
references.citation: Stat Med. 2005 Jul 15;24(13):2025-40
references.pmid: 15900606
references.citation: Science. 1994 Oct 7;266(5182):66-71
references.pmid: 7545954
references.citation: J Clin Pathol. 2009 Jan;62(1):1-5
references.pmid: 18818262
references.citation: Metabolomics. 2012 Jun;8(Suppl 1):3-16
references.pmid: 22593721
references.citation: Nat Biotechnol. 2009 Dec;27(12):1135-7
references.pmid: 20010596