title: OrderRex: clinical order decision support and outcome predictions by data-mining electronic medical records.
abstract: OBJECTIVE
  To answer a "grand challenge" in clinical decision support, the authors produced a recommender system that automatically data-mines inpatient decision support from electronic medical records (EMR), analogous to Netflix or Amazon.com's product recommender.
  
  
  METHODS
  EMR data were extracted from 1 year of hospitalizations (>18K patients with >5.4M structured items including clinical orders, lab results, and diagnosis codes). Association statistics were counted for the ∼1.5K most common items to drive an order recommender. The authors assessed the recommender's ability to predict hospital admission orders and outcomes based on initial encounter data from separate validation patients.
  
  
  RESULTS
  Compared to a reference benchmark of using the overall most common orders, the recommender using temporal relationships improves precision at 10 recommendations from 33% to 38% (P < 10(-10)) for hospital admission orders. Relative risk-based association methods improve inverse frequency weighted recall from 4% to 16% (P < 10(-16)). The framework yields a prediction receiver operating characteristic area under curve (c-statistic) of 0.84 for 30 day mortality, 0.84 for 1 week need for ICU life support, 0.80 for 1 week hospital discharge, and 0.68 for 30-day readmission.
  
  
  CONCLUSIONS
  Recommender results quantitatively improve on reference benchmarks and qualitatively appear clinically reasonable. The method assumes that aggregate decision making converges appropriately, but ongoing evaluation is necessary to discern common behaviors from "correct" ones.
  
  
  CONCLUSIONS
  Collaborative filtering recommender algorithms generate clinical decision support that is predictive of real practice patterns and clinical outcomes. Incorporating temporal relationships improves accuracy. Different evaluation metrics satisfy different goals (predicting likely events vs. "interesting" suggestions).
authors.affiliation: Center for Innovation to Implementation (Ci2i), Veterans Affairs Palo Alto Health Care System, Palo Alto, CA, USA Center for Primary Care and Outcomes Research (PCOR), Stanford University, Stanford, CA, USA.
authors.forename: Jonathan H
authors.initials: JH
authors.lastname: Chen
authors.affiliation: Biomedical Informatics Training Program, Stanford University, Stanford, CA, USA.
authors.forename: Tanya
authors.initials: T
authors.lastname: Podchiyska
authors.affiliation: Departments of Bioengineering, Genetics, and Medicine, Stanford University, Stanford, CA, USA russ.altman@stanford.edu.
authors.forename: Russ B
authors.initials: RB
authors.lastname: Altman
mesh_terms: D000465:Algorithms; D019985:Benchmarking; D057225:Data Mining; D003658:Decision Making, Computer-Assisted; D020000:Decision Support Systems, Clinical; D057286:Electronic Health Records; D006801:Humans; D010343:Patient Admission
keywords: clinical decision support systems; collaborative filtering; data-mining; electronic health records; order sets; practice guidelines; practice variability; recommender algorithms