title: Different methods of balancing covariates leading to different effect estimates in the presence of effect modification.
abstract: A number of covariate-balancing methods, based on the propensity score, are widely used to estimate treatment effects in observational studies. If the treatment effect varies with the propensity score, however, different methods can give very different answers. The authors illustrate this effect by using data from a United Kingdom-based registry of subjects treated with anti-tumor necrosis factor drugs for rheumatoid arthritis. Estimates of the effect of these drugs on mortality varied from a relative risk of 0.4 (95% confidence interval: 0.16, 0.91) to a relative risk of 1.3 (95% confidence interval: 0.8, 2.25), depending on the balancing method chosen. The authors show that these differences were due to a combination of an interaction between propensity score and treatment effect and to differences in weighting subjects with different propensity scores. Thus, the methods are being used to calculate average treatment effects in populations with very different distributions of effect-modifying variables, resulting in different overall estimates. This phenomenon highlights the importance of careful selection of the covariate-balancing method so that the overall estimate has a meaningful interpretation.
authors.affiliation: arc Epidemiology Unit, University of Manchester, Stopford Building, Oxford Road, Manchester, United Kingdom. mark.lunt@manchester.ac.uk
authors.forename: Mark
authors.initials: M
authors.lastname: Lunt
authors.forename: Daniel
authors.initials: D
authors.lastname: Solomon
authors.forename: Kenneth
authors.initials: K
authors.lastname: Rothman
authors.forename: Robert
authors.initials: R
authors.lastname: Glynn
authors.forename: Kimme
authors.initials: K
authors.lastname: Hyrich
authors.forename: Deborah P M
authors.initials: DP
authors.lastname: Symmons
authors.forename: Til
authors.initials: T
authors.lastname: Stürmer
mesh_terms: D000368:Aged; D018501:Antirheumatic Agents / Q000627:therapeutic use; D001172:Arthritis, Rheumatoid / Q000150:complications / Q000188:drug therapy / Q000503:physiopathology; D003627:Data Interpretation, Statistical; D016208:Databases, Factual; D015987:Effect Modifier, Epidemiologic; D015991:Epidemiologic Measurements; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D009026:Mortality; D012306:Risk; D014409:Tumor Necrosis Factor-alpha / Q000037:antagonists & inhibitors
chemical_list: D018501:Antirheumatic Agents; D014409:Tumor Necrosis Factor-alpha