Risk adjustment in the American College of Surgeons National Surgical Quality Improvement Program: a comparison of logistic versus hierarchical modeling.
BACKGROUND
Although logistic regression has commonly been used to adjust for risk differences in patient and case mix to permit quality comparisons across hospitals, hierarchical modeling has been advocated as the preferred methodology, because it accounts for clustering of patients within hospitals. It is unclear whether hierarchical models would yield important differences in quality assessments compared with logistic models when applied to American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP) data. Our objective was to evaluate differences in logistic versus hierarchical modeling for identifying hospitals with outlying outcomes in the ACS-NSQIP.


METHODS
Data from ACS-NSQIP patients who underwent colorectal operations in 2008 at hospitals that reported at least 100 operations were used to generate logistic and hierarchical prediction models for 30-day morbidity and mortality. Differences in risk-adjusted performance (ratio of observed-to-expected events) and outlier detections from the two models were compared.


RESULTS
Logistic and hierarchical models identified the same 25 hospitals as morbidity outliers (14 low and 11 high outliers), but the hierarchical model identified 2 additional high outliers. Both models identified the same eight hospitals as mortality outliers (five low and three high outliers). The values of observed-to-expected events ratios and p values from the two models were highly correlated. Results were similar when data were permitted from hospitals providing < 100 patients.


CONCLUSIONS
When applied to ACS-NSQIP data, logistic and hierarchical models provided nearly identical results with respect to identification of hospitals' observed-to-expected events ratio outliers. As hierarchical models are prone to implementation problems, logistic regression will remain an accurate and efficient method for performing risk adjustment of hospital quality comparisons.
Cohen
Mark E
ME
Division of Research and Optimal Patient Care, American College of Surgeons, 633 N Saint Clair St, 22nd Fl, Chicago, IL 60611-3211, USA. markcohen@facs.org
Dimick
Justin B
JB
Bilimoria
Karl Y
KY
Ko
Clifford Y
CY
Richards
Karen
K
Hall
Bruce Lee
BL
D000368:Aged; D000369:Aged, 80 and over; D003082:Colectomy / Q000592:standards / Q000706:statistics & numerical data*; D005260:Female; D006801:Humans; D007410:Intestinal Diseases / Q000601:surgery*; D016015:Logistic Models; D008297:Male; D008875:Middle Aged; D008962:Models, Theoretical; D017063:Outcome Assessment, Health Care; D015397:Program Evaluation; D011785:Quality Assurance, Health Care; D011787:Quality of Health Care; D020379:Risk Adjustment; D014481:United States