title: Mortality predictions in the intensive care unit: comparing physicians with scoring systems.
abstract: OBJECTIVE
  Risk-prediction models offer potential advantages over physician predictions of outcomes in the intensive care unit (ICU). Our systematic review compared the accuracy of ICU physicians' and scoring system predictions of ICU or hospital mortality of critically ill adults.
  
  
  METHODS
  MEDLINE (1966-2005), CINAHL (1982-2005), Ovid Healthstar (1975-2004), EMBASE (1980-2005), SciSearch (1980-2005), PsychLit (1985-2004), the Cochrane Library (Issue 1, 2005), PubMed "related articles," personal files, abstract proceedings, and reference lists.
  
  
  METHODS
  We considered all studies that compared physician predictions of ICU or hospital survival of critically ill adults to an objective scoring system, computer model, or prediction rule. We excluded studies if they focused exclusively on the development or economic evaluation of a scoring system, computer model, or prediction rule.
  
  
  METHODS
  We independently abstracted data and assessed study quality in duplicate. We determined summary receiver operating characteristic curves and areas under the summary receiver operating characteristic curves+/-se and summary diagnostic odds ratios.
  
  
  RESULTS
  We included 12 observational studies of moderate methodological quality. The area under the summary receiver operating characteristic curves for seven studies was 0.85+/-0.03 for physician predictions compared with 0.63+/-0.06 for scoring system predictions (p=.002). Physicians' summary diagnostic odds ratios derived from the area under the summary receiver operating characteristic curves were significantly higher (12.43; 95% confidence interval 5.47, 27.11) than scoring systems' summary diagnostic odds ratios (2.25; 95% confidence interval 0.78, 6.52, p=.001). Combined results of all 12 studies indicated that physicians predict mortality more accurately than do scoring systems: ratio of diagnostic odds ratios (95% confidence interval) 1.92 (1.19, 3.08) (p=.007).
  
  
  CONCLUSIONS
  Observational studies suggest that ICU physicians discriminate between survivors and nonsurvivors more accurately than do scoring systems in the first 24 hrs of ICU admission. The overall accuracy of both predictions of patient mortality was moderate, implying limited usefulness of outcome prediction in the first 24 hrs for clinical decision making.
authors.affiliation: Department of Critical Care Medicine, Sunnybrook and Women's College Health Sciences Centre, Toronto, Ontario, Canada.
authors.forename: Tasnim
authors.initials: T
authors.lastname: Sinuff
authors.forename: Neill K J
authors.initials: NK
authors.lastname: Adhikari
authors.forename: Deborah J
authors.initials: DJ
authors.lastname: Cook
authors.forename: Holger J
authors.initials: HJ
authors.lastname: Schünemann
authors.forename: Lauren E
authors.initials: LE
authors.lastname: Griffith
authors.forename: Graeme
authors.initials: G
authors.lastname: Rocker
authors.forename: Stephen D
authors.initials: SD
authors.lastname: Walter
mesh_terms: D016638:Critical Illness / Q000401:mortality*; D003661:Decision Support Techniques; D006305:Health Status Indicators; D017052:Hospital Mortality; D006801:Humans; D007362:Intensive Care Units; D007600:Judgment; D010820:Physicians; D012372:ROC Curve; D012044:Regression Analysis; D018570:Risk Assessment; D012680:Sensitivity and Specificity