Different methods of balancing covariates leading to different effect estimates in the presence of effect modification.
A number of covariate-balancing methods, based on the propensity score, are widely used to estimate treatment effects in observational studies. If the treatment effect varies with the propensity score, however, different methods can give very different answers. The authors illustrate this effect by using data from a United Kingdom-based registry of subjects treated with anti-tumor necrosis factor drugs for rheumatoid arthritis. Estimates of the effect of these drugs on mortality varied from a relative risk of 0.4 (95% confidence interval: 0.16, 0.91) to a relative risk of 1.3 (95% confidence interval: 0.8, 2.25), depending on the balancing method chosen. The authors show that these differences were due to a combination of an interaction between propensity score and treatment effect and to differences in weighting subjects with different propensity scores. Thus, the methods are being used to calculate average treatment effects in populations with very different distributions of effect-modifying variables, resulting in different overall estimates. This phenomenon highlights the importance of careful selection of the covariate-balancing method so that the overall estimate has a meaningful interpretation.
Lunt
Mark
M
arc Epidemiology Unit, University of Manchester, Stopford Building, Oxford Road, Manchester, United Kingdom. mark.lunt@manchester.ac.uk
Solomon
Daniel
D
Rothman
Kenneth
K
Glynn
Robert
R
Hyrich
Kimme
K
Symmons
Deborah P M
DP
Stürmer
Til
T
D000368:Aged; D018501:Antirheumatic Agents / Q000627:therapeutic use; D001172:Arthritis, Rheumatoid / Q000150:complications / Q000188:drug therapy / Q000503:physiopathology; D003627:Data Interpretation, Statistical; D016208:Databases, Factual; D015987:Effect Modifier, Epidemiologic; D015991:Epidemiologic Measurements; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D009026:Mortality; D012306:Risk; D014409:Tumor Necrosis Factor-alpha / Q000037:antagonists & inhibitors
D018501:Antirheumatic Agents; D014409:Tumor Necrosis Factor-alpha