title: Comparison of eight computer programs for receiver-operating characteristic analysis.
abstract: BACKGROUND
  ROC analysis is widely accepted to assess and compare diagnostic validity of laboratory tests. Within the last few years, many new ROC programs have become available but have not been systematically evaluated. The aim of this study was to assess different ROC programs regarding their ease of use, mathematical correctness, final output, and their compatibility with other graphics programs.
  
  
  METHODS
  Eight available programs running under Windows (AccuROC, Analyse-It, CMDT, GraphROC, MedCalc, mROC, ROCKIT, and SPSS) were evaluated. ROC analyses of prostate-specific antigen and related values were performed from a dataset of 928 men with prostate cancer and benign prostatic hyperplasia and corresponding subsets. Criteria such as data input, data output, and correctness and completeness of results were used to evaluate the practicability of the programs.
  
  
  RESULTS
  Although the programs produced equivalent results (areas under the curves and their characteristics), we observed deficiencies concerning input of data, processing of the output data, and completeness of the results. Analyse-It, AccuROC, and MedCalc exhibited good performance, but each program had different shortcomings. Only GraphROC could compare curves at a certain sensitivity or specificity cutoff.
  
  
  CONCLUSIONS
  Adequate ROC analysis and ROC plotting cannot be performed with a single program. Analyse-It, AccuROC, and MedCalc can be recommended with certain limitations. Further improvements of the programs are necessary.
authors.affiliation: Department of Urology, University Hospital Charité, Humboldt University, D-10098 Berlin, Germany.
authors.forename: Carsten
authors.initials: C
authors.lastname: Stephan
authors.forename: Sebastian
authors.initials: S
authors.lastname: Wesseling
authors.forename: Tania
authors.initials: T
authors.lastname: Schink
authors.forename: Klaus
authors.initials: K
authors.lastname: Jung
mesh_terms: D019540:Area Under Curve; D006801:Humans; D008297:Male; D016571:Neural Networks, Computer; D017430:Prostate-Specific Antigen / Q000032:analysis; D011470:Prostatic Hyperplasia / Q000175:diagnosis; D011471:Prostatic Neoplasms / Q000175:diagnosis; D012372:ROC Curve; D012984:Software
chemical_list: D017430:Prostate-Specific Antigen