Limitations of the facility-specific standardized mortality ratio for profiling health care quality in dialysis.
Health care quality is assessed by profiling measures of care and/or health outcomes. However, such tools to measure outcome as standardized mortality ratio (SMR) are often used without thorough validation of their strengths and limitations. Our study compared the dialysis facility-specific SMR and SMR-based rating using different statistical methods and followed them over time. All Fresenius Medical Care, North America dialysis facilities (n = 377) that contributed patient data from 1993 to 1995 (>103,500 patient-years) were included. Four distinct statistical methods (US Renal Data System [USRDS], Poisson, logistic, and Cox regression) were used to compute facility-specific SMRs and rank and classify facilities. The analysis compared the SMR and SMR-based rating of dialysis facilities between SMR method and over time. Different methods produced statistically significant differences in SMR distribution (P < 0.05). The USRDS method produced SMR values that decreased over time (P < 0.001). Based on 90% confidence intervals to determine outliers, the SMR-dependent ranking of dialysis facilities varied by method (P < 0.001). SMR-based ranking was stable over time except for the USRDS method (P < 0.001). Contingency table analysis showed up to a 33% total misclassification rate between SMR methods when ranking facilities. The facility-specific SMR and SMR-based ranking are both sensitive to statistical technique. Because the SMR yields different results in a year and over time and because there is no demonstrable gold standard, conclusions based on any one technique are unstable and unreliable. Regulatory monitoring, actions, and/or performance awards should be avoided based on this measure. However, a facility-specific SMR estimated in any valid way may be useful as an epidemiological research tool.
Lacson
E
E
Home Dialysis Therapies and Health Care Information Services, Dialysis Services, Fresenius Medical Care, North America, Lexington, MA, USA. elacsonj@fmc-na.com
Teng
M
M
Lazarus
J M
JM
Lew
N
N
Lowrie
E
E
Owen
W
W
D000328:Adult; D000554:Ambulatory Care Facilities / Q000706:statistics & numerical data*; D003627:Data Interpretation, Statistical; D005260:Female; D006302:Health Services Research; D006801:Humans; D007676:Kidney Failure, Chronic / Q000401:mortality / Q000628:therapy; D008297:Male; D009656:North America; D011787:Quality of Health Care; D012044:Regression Analysis; D006435:Renal Dialysis / Q000401:mortality*; D017749:Total Quality Management