Evaluating trauma center quality: does the choice of the severity-adjustment model make a difference?
BACKGROUND
The Major Trauma Outcome Study (MTOS) database was created by the American College of Surgeons over 20 years ago to establish national norms for trauma care. The primary trauma outcome prediction models used for evaluating the quality of trauma care, TRISS and ASCOT (A Severity Characterization of Trauma), were developed using the MTOS database.


OBJECTIVE
First, to determine whether TRISS and ASCOT agree on hospital quality. Second, to determine whether TRISS and ASCOT accurately reflect contemporary outcomes in trauma care.


METHODS
A retrospective cohort study based on 91,112 patients admitted to 69 hospitals between 2000 and 2001 in the National Trauma Databank. Using TRISS and ASCOT, the ratio of the observed to expected mortality rate (O/E ratio) was calculated for each hospital. Hospitals whose O/E ratio was statistically different from 1 were identified as quality outliers. Kappa analysis was used to assess the degree to which TRISS and ASCOT agreed on the identity of hospital quality outliers.


RESULTS
TRISS and ASCOT disagreed on the outlier status of 35 of the 69 hospitals. Kappa analysis revealed only fair agreement (kappa = 0.23; p = 0.0015) between TRISS and ASCOT in identifying quality outliers. Thirty-eight hospitals were identified by the TRISS method as high-performance hospitals.


CONCLUSIONS
First, TRISS and ASCOT exhibit substantial disagreement on the identity of quality outliers within the NTDB. Second, an unrealistically high number of hospitals were identified as high-performance outliers using either TRISS or ASCOT. These findings have important implications for the use of TRISS and ASCOT for benchmarking performance and quality improvement.
Glance
Laurent G
LG
Department of Anesthesiology, University of Rochester School of Medicine and Dentistry , NY 14642, USA. Laurent_Glance@urmc.rochester.edu
Osler
Turner M
TM
Dick
Andrew W
AW
D002138:Calibration; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D017063:Outcome Assessment, Health Care; D016311:Outliers, DRG; D011787:Quality of Health Care; D014193:Trauma Centers / Q000145:classification / Q000592:standards* / Q000706:statistics & numerical data; D015599:Trauma Severity Indices