title: A three-step approach for the derivation and validation of high-performing predictive models using an operational dataset: congestive heart failure readmission case study.
abstract: BACKGROUND
  The aim of this study was to propose an analytical approach to develop high-performing predictive models for congestive heart failure (CHF) readmission using an operational dataset with incomplete records and changing data over time.
  
  
  METHODS
  Our analytical approach involves three steps: pre-processing, systematic model development, and risk factor analysis. For pre-processing, variables that were absent in >50% of records were removed. Moreover, the dataset was divided into a validation dataset and derivation datasets which were separated into three temporal subsets based on changes to the data over time. For systematic model development, using the different temporal datasets and the remaining explanatory variables, the models were developed by combining the use of various (i) statistical analyses to explore the relationships between the validation and the derivation datasets; (ii) adjustment methods for handling missing values; (iii) classifiers; (iv) feature selection methods; and (iv) discretization methods. We then selected the best derivation dataset and the models with the highest predictive performance. For risk factor analysis, factors in the highest-performing predictive models were analyzed and ranked using (i) statistical analyses of the best derivation dataset, (ii) feature rankers, and (iii) a newly developed algorithm to categorize risk factors as being strong, regular, or weak.
  
  
  RESULTS
  The analysis dataset consisted of 2,787 CHF hospitalizations at University of Utah Health Care from January 2003 to June 2013. In this study, we used the complete-case analysis and mean-based imputation adjustment methods; the wrapper subset feature selection method; and four ranking strategies based on information gain, gain ratio, symmetrical uncertainty, and wrapper subset feature evaluators. The best-performing models resulted from the use of a complete-case analysis derivation dataset combined with the Class-Attribute Contingency Coefficient discretization method and a voting classifier which averaged the results of multi-nominal logistic regression and voting feature intervals classifiers. Of 42 final model risk factors, discharge disposition, discretized age, and indicators of anemia were the most significant. This model achieved a c-statistic of 86.8%.
  
  
  CONCLUSIONS
  The proposed three-step analytical approach enhanced predictive model performance for CHF readmissions. It could potentially be leveraged to improve predictive model performance in other areas of clinical medicine.
authors.affiliation: Department of Biomedical Informatics, University of Utah, 615 Arapeen Way, Suite 208, Salt Lake City, UT 84092, USA. samir.abdelrahman@utah.edu.
authors.forename: Samir E
authors.initials: SE
authors.lastname: AbdelRahman
authors.forename: Mingyuan
authors.initials: M
authors.lastname: Zhang
authors.forename: Bruce E
authors.initials: BE
authors.lastname: Bray
authors.forename: Kensaku
authors.initials: K
authors.lastname: Kawamoto
mesh_terms: D000046:Academic Medical Centers; D006333:Heart Failure / Q000628:therapy*; D006760:Hospitalization; D006801:Humans; D015233:Models, Statistical; D010359:Patient Readmission; D011237:Predictive Value of Tests; D015203:Reproducibility of Results; D012307:Risk Factors; D063128:Tertiary Health Care