title: Assessing and combining repeated prognosis of physicians and temporal models in the intensive care.
abstract: OBJECTIVE
  Recently, we devised a method to develop prognostic models incorporating patterns of sequential organ failure to predict the eventual hospital mortality at each day of intensive care unit (ICU) stay. In this study, we investigate using a real world setting how these models perform compared to physicians, who are exposed to additional information than the models.
  
  
  METHODS
  We developed prognostic models for days 2-7 of ICU stay by data-driven discovery of patterns of sequential qualitative organ failure (SOFA) scores and embedding the patterns as binary variables in three types of logistic regression models. Type A models include the severity of illness score at admission (SAPS-II) and the SOFA patterns. Type B models add to these covariates the mean, max and delta (increments) of SOFA scores. Type C models include, in addition, the mean, max and delta in expert opinion (i.e. the physicians' prediction of mortality).
  
  
  RESULTS
  Physicians had a statistically significantly better discriminative ability compared to the models without subjective information (AUC range over days: 0.78-0.79 vs. 0.71-0.74) and comparable accuracy (Brier score range: 0.15-0.18 vs. 0.16-0.18). However when we combined both sources of predictions, in Type C models, we arrived at a significantly superior discrimination as well as accuracy than the objective and subjective models alone (AUC range: 0.80-0.83; Brier score range: 0.13-0.16).
  
  
  CONCLUSIONS
  The models and the physicians draw on complementary information that can be best harnessed by combining both prediction sources. Extensive external validation and impact studies are imperative to further investigate the ability of the combined model.
authors.affiliation: Department of Medical Informatics, Academic Medical Center, PO Box 22660, 1100 DD Amsterdam, The Netherlands. L.Minne@amc.uva.nl
authors.forename: Lilian
authors.initials: L
authors.lastname: Minne
authors.forename: Tudor
authors.initials: T
authors.lastname: Toma
authors.forename: Evert
authors.initials: E
authors.lastname: de Jonge
authors.forename: Ameen
authors.initials: A
authors.lastname: Abu-Hanna
mesh_terms: D000328:Adult; D000367:Age Factors; D000368:Aged; D003198:Computer Simulation; D005260:Female; D006305:Health Status Indicators; D017052:Hospital Mortality; D006785:Hospitals, University; D006801:Humans; D007362:Intensive Care Units / Q000706:statistics & numerical data*; D016015:Logistic Models; D008297:Male; D008875:Middle Aged; D063305:Organ Dysfunction Scores; D010820:Physicians; D011379:Prognosis; D011446:Prospective Studies; D012737:Sex Factors; D013997:Time Factors