title: Comparison of the Mortality Probability Admission Model III, National Quality Forum, and Acute Physiology and Chronic Health Evaluation IV hospital mortality models: implications for national benchmarking*.
abstract: OBJECTIVE
  To examine the accuracy of the original Mortality Probability Admission Model III, ICU Outcomes Model/National Quality Forum modification of Mortality Probability Admission Model III, and Acute Physiology and Chronic Health Evaluation IVa models for comparing observed and risk-adjusted hospital mortality predictions.
  
  
  METHODS
  Retrospective paired analyses of day 1 hospital mortality predictions using three prognostic models.
  
  
  METHODS
  Fifty-five ICUs at 38 U.S. hospitals from January 2008 to December 2012.
  
  
  METHODS
  Among 174,001 intensive care admissions, 109,926 met model inclusion criteria and 55,304 had data for mortality prediction using all three models.
  
  
  METHODS
  None.
  
  
  RESULTS
  We compared patient exclusions and the discrimination, calibration, and accuracy for each model. Acute Physiology and Chronic Health Evaluation IVa excluded 10.7% of all patients, ICU Outcomes Model/National Quality Forum 20.1%, and Mortality Probability Admission Model III 24.1%. Discrimination of Acute Physiology and Chronic Health Evaluation IVa was superior with area under receiver operating curve (0.88) compared with Mortality Probability Admission Model III (0.81) and ICU Outcomes Model/National Quality Forum (0.80). Acute Physiology and Chronic Health Evaluation IVa was better calibrated (lowest Hosmer-Lemeshow statistic). The accuracy of Acute Physiology and Chronic Health Evaluation IVa was superior (adjusted Brier score = 31.0%) to that for Mortality Probability Admission Model III (16.1%) and ICU Outcomes Model/National Quality Forum (17.8%). Compared with observed mortality, Acute Physiology and Chronic Health Evaluation IVa overpredicted mortality by 1.5% and Mortality Probability Admission Model III by 3.1%; ICU Outcomes Model/National Quality Forum underpredicted mortality by 1.2%. Calibration curves showed that Acute Physiology and Chronic Health Evaluation performed well over the entire risk range, unlike the Mortality Probability Admission Model and ICU Outcomes Model/National Quality Forum models. Acute Physiology and Chronic Health Evaluation IVa had better accuracy within patient subgroups and for specific admission diagnoses.
  
  
  CONCLUSIONS
  Acute Physiology and Chronic Health Evaluation IVa offered the best discrimination and calibration on a large common dataset and excluded fewer patients than Mortality Probability Admission Model III or ICU Outcomes Model/National Quality Forum. The choice of ICU performance benchmarks should be based on a comparison of model accuracy using data for identical patients.
authors.affiliation: 1Cerner Corporation, Vienna, VA. 2Department of Biostatistics, Kansas University Medical Center, Kansas City, MO. 3Critical Care Division, Baystate Medical Center, Springfield, MA. 4Department of Medicine, Tufts University School of Medicine, Boston, MA. 5Department of Anesthesiology and Critical Care Medicine, George Washington University, Washington, DC.
authors.forename: Andrew A
authors.initials: AA
authors.lastname: Kramer
authors.forename: Thomas L
authors.initials: TL
authors.lastname: Higgins
authors.forename: Jack E
authors.initials: JE
authors.lastname: Zimmerman
mesh_terms: D000368:Aged; D019985:Benchmarking; D016638:Critical Illness / Q000401:mortality / Q000628:therapy; D016208:Databases, Factual; D005260:Female; D017052:Hospital Mortality / Q000639:trends*; D006801:Humans; D007362:Intensive Care Units; D008297:Male; D008875:Middle Aged; D015233:Models, Statistical; D008962:Models, Theoretical; D010343:Patient Admission / Q000706:statistics & numerical data*; D011237:Predictive Value of Tests; D011336:Probability; D011379:Prognosis; D019984:Quality Indicators, Health Care; D012189:Retrospective Studies; D014481:United States