title: Testing the incremental predictive accuracy of new markers.
abstract: BACKGROUND
  It has become commonplace to use receiver operating curve (ROC) methodology to evaluate the incremental predictive accuracy of new markers in the presence of existing predictors. However, concerns have been raised about the validity of this practice. We have evaluated this issue in detail.
  
  
  RESULTS
  Simulations have been used that show clearly that use of risk predictors from nested models as data in subsequent tests comparing areas under the ROC curves of the models leads to grossly invalid inferences. Careful examination of the issue reveals two major problems: (1) the data elements are strongly correlated from case to case and (2) the model that includes the additional marker has a tendency to interpret predictive contributions as positive information regardless of whether observed effect of the marker is negative or positive. Both of these phenomena lead to profound bias in the test.
  
  
  CONCLUSIONS
  We recommend strongly against the use of ROC methods derived from risk predictors from nested regression models to test the incremental information of a new marker.
authors.affiliation: Department of Epidemiology and Biostatistics, Memorial Sloan-Kettering Cancer Center, New York, USA.
authors.forename: Colin B
authors.initials: CB
authors.lastname: Begg
authors.forename: Mithat
authors.initials: M
authors.lastname: Gonen
authors.forename: Venkatraman E
authors.initials: VE
authors.lastname: Seshan
mesh_terms: D015415:Biomarkers; D002986:Clinical Trials as Topic / Q000379:methods; D006801:Humans; D011237:Predictive Value of Tests; D012372:ROC Curve; D015203:Reproducibility of Results; D018570:Risk Assessment
chemical_list: D015415:Biomarkers