Predicting risk-adjusted mortality for CABG surgery: logistic versus hierarchical logistic models.
BACKGROUND
In recent years, several studies in the medical and health service research literature have advocated the use of hierarchical statistical models (multilevel models or random-effects models) to analyze data that are nested (eg, patients nested within hospitals). However, these models are computer-intensive and complicated to perform. There is virtually nothing in the literature that compares the results of standard logistic regression to those of hierarchical logistic models in predicting future provider performance.


OBJECTIVE
We sought to compare the ability of standard logistic regression relative to hierarchical modeling in predicting risk-adjusted hospital mortality rates for coronary artery bypass graft (CABG) surgery in New York State.


METHODS
New York State CABG Registry data from 1994 to 1999 were used to relate statistical predictions from a given year to hospital performance 2 years hence.


METHODS
Predicted and observed hospital mortality rates 2 years hence were compared using root mean square errors, the mean absolute difference, and the number of hospitals whose predicted mortality rate data was within a 95% confidence interval around the observed mortality rate.


RESULTS
In these data, standard logistic regression performed similarly to hierarchical models, both with and without a second level covariate. Differences in the criteria used for comparison were minimal, and when the differences could be statistically tested no significant differences were identified.


CONCLUSIONS
It is instructive to compare the predictive abilities of alternative statistical models in the process of assessing their relative performance on a specific database and application.
Hannan
Edward L
EL
Department of Health Policy, Management, and Behavior, School of Public Health, University at Albany, State University of New York, Albany, New York 12144-3456, USA. elh03@health.state.ny.us
Wu
Chuntao
C
DeLong
Elizabeth R
ER
Raudenbush
Stephen W
SW
D001026:Coronary Artery Bypass / Q000401:mortality*; D017052:Hospital Mortality; D006801:Humans; D016015:Logistic Models; D015233:Models, Statistical; D009518:New York; D011237:Predictive Value of Tests