A minimal net reclassification improvement to assess predictions of intensive care mortality.
OBJECTIVE
In assessing the improved discrimination of a new prognostic score, the "Net Reclassification Improvement" from reclassification methods appears of interest. We propose a measure that takes into account improvements in predicted probabilities to assess and allows testing the additional predictive ability of a new scoring system Y compared to a reference score X.


METHODS
To assess and test the improvement in mortality prediction of (X + Y) compared to X, we defined a minimal net reclassification improvement that restricted improvements in predicted probabilities according to some positive threshold δ. Both absolute and relative improvements were considered. A simulation study was performed to assess its performances in a range of practical situations. We then applied our measures to real intensive care unit data.


RESULTS
Expectedly, minimal net reclassification improvement increased with the effect size of Y and decreased with the value of δ. Using relative improvements allowed erasing the influence of the population mortality. For given effect sizes of X and Y, the difference in all measures of reclassification decreased when a correlation between X and Y was introduced.


CONCLUSIONS
Reclassification methods, particularly the minimal net reclassification improvement, seem to be clinically relevant when used with continuous clinical data with no known threshold.
INSERM, U717, Paris, F-75010, France Université Paris Diderot, PRES Sorbonne Paris Cité, UMR-S717, Paris, F-75010, France AP-HP Paris, F-75010, France.
Yacine Tandjaoui Lambiotte
YT
Redondo
INSERM, U717, Paris, F-75010, France Université Paris Diderot, PRES Sorbonne Paris Cité, UMR-S717, Paris, F-75010, France AP-HP Paris, F-75010, France.
Jérôme
J
Lambert
INSERM, U717, Paris, F-75010, France Université Paris Diderot, PRES Sorbonne Paris Cité, UMR-S717, Paris, F-75010, France AP-HP Paris, F-75010, France sylvie.chevret@paris7.jussieu.fr.
Sylvie
S
Chevret
D056808:Biostatistics; D003198:Computer Simulation; D003422:Critical Care; D017052:Hospital Mortality; D006801:Humans; D015233:Models, Statistical; D011336:Probability; D011379:Prognosis; D012131:Respiratory Insufficiency / Q000401:mortality; D012307:Risk Factors