title: pROC: an open-source package for R and S+ to analyze and compare ROC curves.
abstract: BACKGROUND
  Receiver operating characteristic (ROC) curves are useful tools to evaluate classifiers in biomedical and bioinformatics applications. However, conclusions are often reached through inconsistent use or insufficient statistical analysis. To support researchers in their ROC curves analysis we developed pROC, a package for R and S+ that contains a set of tools displaying, analyzing, smoothing and comparing ROC curves in a user-friendly, object-oriented and flexible interface.
  
  
  RESULTS
  With data previously imported into the R or S+ environment, the pROC package builds ROC curves and includes functions for computing confidence intervals, statistical tests for comparing total or partial area under the curve or the operating points of different classifiers, and methods for smoothing ROC curves. Intermediary and final results are visualised in user-friendly interfaces. A case study based on published clinical and biomarker data shows how to perform a typical ROC analysis with pROC.
  
  
  CONCLUSIONS
  pROC is a package for R and S+ specifically dedicated to ROC analysis. It proposes multiple statistical tests to compare ROC curves, and in particular partial areas under the curve, allowing proper ROC interpretation. pROC is available in two versions: in the R programming language or with a graphical user interface in the S+ statistical software. It is accessible at http://expasy.org/tools/pROC/ under the GNU General Public License. It is also distributed through the CRAN and CSAN public repositories, facilitating its installation.
authors.affiliation: Biomedical Proteomics Research Group, Department of Structural Biology and Bioinformatics, Medical University Centre, Geneva, Switzerland. Xavier.Robin@unige.ch
authors.forename: Xavier
authors.initials: X
authors.lastname: Robin
authors.forename: Natacha
authors.initials: N
authors.lastname: Turck
authors.forename: Alexandre
authors.initials: A
authors.lastname: Hainard
authors.forename: Natalia
authors.initials: N
authors.lastname: Tiberti
authors.forename: Frédérique
authors.initials: F
authors.lastname: Lisacek
authors.forename: Jean-Charles
authors.initials: JC
authors.lastname: Sanchez
authors.forename: Markus
authors.initials: M
authors.lastname: Müller
mesh_terms: D015415:Biomarkers / Q000032:analysis; D019295:Computational Biology / Q000379:methods*; D016001:Confidence Intervals; D003627:Data Interpretation, Statistical; D006801:Humans; D011381:Programming Languages; D012372:ROC Curve; D012984:Software
chemical_list: D015415:Biomarkers