title: Evaluation of two outcome prediction models on an independent database.
abstract: OBJECTIVE
  To evaluate the performance of the New Simplified Acute Physiology Score (SAPS II) and the admission Mortality Probability Model (MPM0) in a large independent database, using formal statistical assessment.
  
  
  METHODS
  Analysis of the database of a multicenter, multinational, prospective cohort study, EURICUS-I.
  
  
  METHODS
  Eighty nine intensive care units (ICUs) from 13 European areas.
  
  
  METHODS
  Data of 16,060 patients consecutively admitted to the participating ICUs were collected during a period of 4 months. Following the original SAPS II and MPM0 criteria, the analysis excluded: patients <18 ys of age; readmissions; patients admitted with acute myocardial infarction; burns; and patients in the postoperative period after coronary artery bypass surgery. All patients with a length of stay <8 hrs were excluded from the study to keep comparability between both systems. A total of 10,027 patients were analyzed.
  
  
  METHODS
  Collection of the first 24 hrs' admission data necessary for the calculation of SAPS II and MPM0 and basic demographic statistics. Vital status at discharge from the hospital was registered.
  
  
  RESULTS
  Despite having a good discriminative capability, as measured by the area under the receiver operating characteristic (ROC) curves (SAPS II: ROC = 0.822 +/- 0.005 SEM; MPM0: ROC = 0.785 +/- 0.006 SEM), both models presented poor calibration, with significant differences between observed and predicted mortality (Hosmer-Lemeshow goodness-of-fit tests H and C, p < .0001). Both SAPS II (predicted risk >40%) and MPM0 (predicted risk >30%) overestimated the risk of death. The evaluation of the uniformity of fit of SAPS II and MPM0 demonstrated large variations across the various subgroups of patients.
  
  
  CONCLUSIONS
  The original SAPS II and MPM0 models did not accurately predict mortality on an independent large international multicenter ICU patient database. Results of studies utilizing general outcome prediction models without previous validation in the target population should be interpreted with prudence.
authors.affiliation: Intensive Care Unit, Hospital de Santo António dos Capuchos, Lisboa, Portugal.
authors.forename: R
authors.initials: R
authors.lastname: Moreno
authors.forename: D R
authors.initials: DR
authors.lastname: Miranda
authors.forename: V
authors.initials: V
authors.lastname: Fidler
authors.forename: R
authors.initials: R
authors.lastname: Van Schilfgaarde
mesh_terms: D018806:APACHE; D000293:Adolescent; D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D015331:Cohort Studies; D016638:Critical Illness / Q000401:mortality*; D005060:Europe; D006302:Health Services Research / Q000706:statistics & numerical data; D017052:Hospital Mortality; D006801:Humans; D007362:Intensive Care Units / Q000706:statistics & numerical data*; D008875:Middle Aged; D015233:Models, Statistical; D011237:Predictive Value of Tests; D011336:Probability; D011446:Prospective Studies; D012372:ROC Curve; D012720:Severity of Illness Index; D015996:Survival Rate