title: Treating metastatic disease: Which survival model is best suited for the clinic?
abstract: BACKGROUND
  To avoid complications associated with under- or overtreatment of patients with skeletal metastases, doctors need accurate survival estimates. Unfortunately, prognostic models for patients with skeletal metastases of the extremities are lacking, and physician-based estimates are generally inaccurate.
  
  
  OBJECTIVE
  We developed three types of prognostic models and compared them using calibration plots, receiver operating characteristic (ROC) curves, and decision curve analysis to determine which one is best suited for clinical use.
  
  
  METHODS
  A training set consisted of 189 patients who underwent surgery for skeletal metastases. We created models designed to predict 3- and 12-month survival using three methods: an Artificial Neural Network (ANN), a Bayesian Belief Network (BBN), and logistic regression. We then performed crossvalidation and compared the models in three ways: calibration plots plotting predicted against actual risk; area under the ROC curve (AUC) to discriminate the probability that a patient who died has a higher predicted probability of death compared to a patient who did not die; and decision curve analysis to quantify the clinical consequences of over- or undertreatment.
  
  
  RESULTS
  All models appeared to be well calibrated, with the exception of the BBN, which underestimated 3-month survival at lower probability estimates. The ANN models had the highest discrimination, with an AUC of 0.89 and 0.93, respectively, for the 3- and 12-month models. Decision analysis revealed all models could be used clinically, but the ANN models consistently resulted in the highest net benefit, outperforming the BBN and logistic regression models.
  
  
  CONCLUSIONS
  Our observations suggest use of the ANN model to aid decisions about surgery would lead to better patient outcomes than other alternative approaches to decision making.
  
  
  METHODS
  Level II, prognostic study. See Instructions for Authors for a complete description of levels of evidence.
authors.affiliation: Regenerative Medicine, Naval Medical Research Center, 503 Robert Grant Ave, Silver Spring, MD 20910, USA. jaforsberg@me.com
authors.forename: Jonathan Agner
authors.initials: JA
authors.lastname: Forsberg
authors.forename: Daniel
authors.initials: D
authors.lastname: Sjoberg
authors.forename: Qing-Rong
authors.initials: QR
authors.lastname: Chen
authors.forename: Andrew
authors.initials: A
authors.lastname: Vickers
authors.forename: John H
authors.initials: JH
authors.lastname: Healey
mesh_terms: D000368:Aged; D001499:Bayes Theorem; D001859:Bone Neoplasms / Q000401:mortality / Q000556:secondary* / Q000601:surgery*; D003661:Decision Support Techniques; D005260:Female; D006801:Humans; D016015:Logistic Models; D008297:Male; D008875:Middle Aged; D016571:Neural Networks, Computer; D019637:Orthopedic Procedures / Q000009:adverse effects / Q000401:mortality; D018579:Patient Selection; D011237:Predictive Value of Tests; D011336:Probability; D012372:ROC Curve; D012189:Retrospective Studies; D018570:Risk Assessment; D012307:Risk Factors; D013997:Time Factors; D016896:Treatment Outcome