Do observational studies using propensity score methods agree with randomized trials? A systematic comparison of studies on acute coronary syndromes.
OBJECTIVE
Randomized controlled trials (RCTs) are the gold standard for assessing the efficacy of therapeutic interventions because randomization protects from biases inherent in observational studies. Propensity score (PS) methods, proposed as a potential solution to confounding of the treatment-outcome association, are widely used in observational studies of therapeutic interventions for acute coronary syndromes (ACS). We aimed to systematically assess agreement between observational studies using PS methods and RCTs on therapeutic interventions for ACS.


RESULTS
We searched for observational studies of interventions for ACS that used PS methods to estimate treatment effects on short- or long-term mortality. Using a standardized algorithm, we matched observational studies to RCTs based on patients' characteristics, interventions, and outcomes ('topics'), and we compared estimates of treatment effect between the two designs. When multiple observational studies or RCTs were identified for the same topic, we performed a meta-analysis and used the summary relative risk for comparisons. We matched 21 observational studies investigating 17 distinct clinical topics to 63 RCTs (median = 3 RCTs per observational study) for short-term (7 topics) and long-term (10 topics) mortality. Estimates from PS analyses differed statistically significantly from randomized evidence in two instances; however, observational studies reported more extreme beneficial treatment effects compared with RCTs in 13 of 17 instances (P = 0.049). Sensitivity analyses limited to large RCTs, and using alternative meta-analysis models yielded similar results.


CONCLUSIONS
For the treatment of ACS, observational studies using PS methods produce treatment effect estimates that are of more extreme magnitude compared with those from RCTs, although the differences are rarely statistically significant.
Center for Clinical Evidence Synthesis, Institute for Clinical Research and Health Policy Studies, Tufts Medical Center, 800 Washington Street, Box No 63, Boston, MA 02111, USA. idahabreh@tuftsmedicalcenter.org
Issa J
IJ
Dahabreh
Radley C
RC
Sheldrick
Jessica K
JK
Paulus
Mei
M
Chung
Vasileia
V
Varvarigou
Haseeb
H
Jafri
Jeremy A
JA
Rassen
Thomas A
TA
Trikalinos
Georgios D
GD
Kitsios
D054058:Acute Coronary Syndrome / Q000401:mortality / Q000628:therapy*; D006801:Humans; D057216:Propensity Score; D016032:Randomized Controlled Trials as Topic / Q000706:statistics & numerical data*; D012044:Regression Analysis; D015203:Reproducibility of Results; D012307:Risk Factors; D016896:Treatment Outcome