Bias in estimates of confidence intervals for health outcome report cards.
Health researchers commonly use logistic regression when profiling health providers. Data from the patients treated by the providers are used to construct models predicting the expected number of outcomes for providers and the ratio of observed to expected outcomes (O/E ratio) used as a risk-adjusted measure of provider performance. Typically, when calculating the standard deviation (SD) of O/E ratios, only O is treated as a random variable. We used the propagation of errors (Pe) to derive a SD estimate that accounted for variability in O and the estimate of E. Using data previously used to profile Canadian cardiac surgery providers, we compared Pe-SD estimates with typical SD (SDT) estimates. The SDT estimates and confidence intervals were always larger than the Pe estimates, most notably when one or more providers treated a large proportion of the patients. This was confirmed using computer simulations. SDT estimates should be abandoned in favor of more sophisticated estimates.
Faris
Peter D
PD
Department of Community Health Sciences, University of Calgary, 3330 Hospital Drive NW, Calgary, Alberta, Canada T2N 4N1.
Ghali
William A
WA
Brant
Rollin
R
D015982:Bias; D002170:Canada; D003198:Computer Simulation; D016001:Confidence Intervals; D001026:Coronary Artery Bypass; D005260:Female; D006302:Health Services Research / Q000379:methods*; D017052:Hospital Mortality; D006801:Humans; D016015:Logistic Models; D008297:Male; D016017:Odds Ratio; D017063:Outcome Assessment, Health Care / Q000379:methods*; D020379:Risk Adjustment; D012307:Risk Factors; D016019:Survival Analysis