title: The effects of data sources, cohort selection, and outcome definition on a predictive model of risk of thirty-day hospital readmissions.
abstract: BACKGROUND
  Hospital readmission risk prediction remains a motivated area of investigation and operations in light of the hospital readmissions reduction program through CMS. Multiple models of risk have been reported with variable discriminatory performances, and it remains unclear how design factors affect performance.
  
  
  OBJECTIVE
  To study the effects of varying three factors of model development in the prediction of risk based on health record data: (1) reason for readmission (primary readmission diagnosis); (2) available data and data types (e.g. visit history, laboratory results, etc); (3) cohort selection.
  
  
  METHODS
  Regularized regression (LASSO) to generate predictions of readmissions risk using prevalence sampling. Support Vector Machine (SVM) used for comparison in cohort selection testing. Calibration by model refitting to outcome prevalence.
  
  
  RESULTS
  Predicting readmission risk across multiple reasons for readmission resulted in ROC areas ranging from 0.92 for readmission for congestive heart failure to 0.71 for syncope and 0.68 for all-cause readmission. Visit history and laboratory tests contributed the most predictive value; contributions varied by readmission diagnosis. Cohort definition affected performance for both parametric and nonparametric algorithms. Compared to all patients, limiting the cohort to patients whose index admission and readmission diagnoses matched resulted in a decrease in average ROC from 0.78 to 0.55 (difference in ROC 0.23, p value 0.01). Calibration plots demonstrate good calibration with low mean squared error.
  
  
  CONCLUSIONS
  Targeting reason for readmission in risk prediction impacted discriminatory performance. In general, laboratory data and visit history data contributed the most to prediction; data source contributions varied by reason for readmission. Cohort selection had a large impact on model performance, and these results demonstrate the difficulty of comparing results across different studies of predictive risk modeling.
authors.affiliation: Department of Biomedical Informatics, Columbia University, United States; Department of Medicine, Columbia University, United States. Electronic address: cgw2106@columbia.edu.
authors.forename: Colin
authors.initials: C
authors.lastname: Walsh
authors.affiliation: Department of Biomedical Informatics, Columbia University, United States.
authors.forename: George
authors.initials: G
authors.lastname: Hripcsak
mesh_terms: D000293:Adolescent; D000328:Adult; D000368:Aged; D057225:Data Mining / Q000379:methods*; D057286:Electronic Health Records / Q000706:statistics & numerical data*; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D015233:Models, Statistical; D010359:Patient Readmission / Q000706:statistics & numerical data*; D015203:Reproducibility of Results; D012107:Research Design / Q000706:statistics & numerical data*; D012189:Retrospective Studies; D018570:Risk Assessment; D055815:Young Adult
keywords: Electronic health record; Predictive analytics; Readmissions; Regularized Logistic Regression; Risk modeling; Text mining