title: Independent external validation of predictive models for urinary dysfunction following external beam radiotherapy of the prostate: Issues in model development and reporting.
abstract: Most predictive models are not sufficiently validated for prospective use. We performed independent external validation of published predictive models for urinary dysfunctions following radiotherapy of the prostate.
  
  
  
  Multivariable models developed to predict atomised and generalised urinary symptoms, both acute and late, were considered for validation using a dataset representing 754 participants from the TROG 03.04-RADAR trial. Endpoints and features were harmonised to match the predictive models. The overall performance, calibration and discrimination were assessed.
  
  
  
  14 models from four publications were validated. The discrimination of the predictive models in an independent external validation cohort, measured using the area under the receiver operating characteristic (ROC) curve, ranged from 0.473 to 0.695, generally lower than in internal validation. 4 models had ROC >0.6. Shrinkage was required for all predictive models' coefficients ranging from -0.309 (prediction probability was inverse to observed proportion) to 0.823. Predictive models which include baseline symptoms as a feature produced the highest discrimination. Two models produced a predicted probability of 0 and 1 for all patients.
  
  
  
  Predictive models vary in performance and transferability illustrating the need for improvements in model development and reporting. Several models showed reasonable potential but efforts should be increased to improve performance. Baseline symptoms should always be considered as potential features for predictive models.
authors.affiliation: School of Physics, University of Western Australia, Australia; School of Health Sciences, National University of Malaysia, Malaysia. Electronic address: noorazrul.yahya@research.uwa.edu.au.
authors.forename: Noorazrul
authors.initials: N
authors.lastname: Yahya
authors.affiliation: School of Physics, University of Western Australia, Australia; Department of Radiation Oncology, Sir Charles Gairdner Hospital, Australia.
authors.forename: Martin A
authors.initials: MA
authors.lastname: Ebert
authors.affiliation: Institute for Health Research, University of Notre Dame, Fremantle, Australia.
authors.forename: Max
authors.initials: M
authors.lastname: Bulsara
authors.affiliation: Department of Radiation Oncology, Sir Charles Gairdner Hospital, Australia.
authors.forename: Angel
authors.initials: A
authors.lastname: Kennedy
authors.affiliation: Department of Radiation Oncology, Sir Charles Gairdner Hospital, Australia; School of Surgery, University of Western Australia, Australia.
authors.forename: David J
authors.initials: DJ
authors.lastname: Joseph
authors.affiliation: School of Medicine and Public Health, University of Newcastle, Australia.
authors.forename: James W
authors.initials: JW
authors.lastname: Denham
mesh_terms: D002138:Calibration; D006801:Humans; D059411:Lower Urinary Tract Symptoms / Q000209:etiology* / Q000503:physiopathology; D008297:Male; D008954:Models, Biological; D011237:Predictive Value of Tests; D011336:Probability; D011446:Prospective Studies; D011471:Prostatic Neoplasms / Q000532:radiotherapy*; D012372:ROC Curve; D011832:Radiation Injuries / Q000209:etiology* / Q000503:physiopathology; D011880:Radiotherapy Planning, Computer-Assisted / Q000379:methods*; D015203:Reproducibility of Results; D014551:Urinary Tract / Q000528:radiation effects*
keywords: Independent external validation; Normal tissue complications; Predictive model; Prostate radiotherapy; Urinary symptoms