title: Can We Trust Observational Studies Using Propensity Scores in the Critical Care Literature? A Systematic Comparison With Randomized Clinical Trials.
abstract: OBJECTIVE
  To assess the degree of agreement between propensity score studies and randomized clinical trials in critical care research.
  
  
  METHODS
  Propensity score studies published in highly cited critical care or general medicine journals or included in a previous systematic review; corresponding randomized clinical trials included in Cochrane Systematic Reviews or published in PubMed.
  
  
  METHODS
  We identified propensity score studies of the effects of therapeutic interventions on short- or long-term mortality. We systematically matched propensity score studies to randomized clinical trials based on patient selection criteria, interventions, and outcomes.
  
  
  METHODS
  We appraised the methods of included studies and extracted treatment effect estimates to compare the results of propensity score studies and randomized clinical trials. When multiple studies were identified for the same topic, we performed meta-analyses to obtain summary treatment effect estimates.
  
  
  RESULTS
  We matched 21 propensity score studies with 58 randomized clinical trials in 18 distinct comparisons (median, one propensity score study and two randomized clinical trials per comparison), for short- and long-term mortality. We found one statistically significant difference between designs (hyperoncotic albumin vs crystalloid fluids) among these 18 comparisons. Propensity score studies did not produce systematically higher (or lower) treatment effect estimates compared with randomized clinical trials, but estimates from the two designs differed by more than 30% in one third of the comparisons examined. Observational studies in critical care met widely accepted methodological standards for propensity score analyses.
  
  
  CONCLUSIONS
  Across diverse critical care topics, propensity score studies published in high-impact journals produced results that were generally consistent with the findings of randomized clinical trials. However, caution is needed when interpreting propensity score studies because occasionally their results contradict those of randomized clinical trials and there is no reliable way to predict disagreements.
authors.affiliation: 1Division of Pulmonary, Allergy, and Critical Care Medicine, University of Pittsburgh Medical Center, Pittsburgh, PA. 2Center for Evidence-Based Medicine, School of Public Health, Brown University, Providence, RI. 3Department of Health Services, Policy & Practice, School of Public Health, Brown University, Providence, RI. 4Division Pulmonary, Critical Care, Allergy, and Sleep, Medical University of South Carolina, Charleston, SC. 5Predictive Analytics and Comparative Effectiveness Center, Institute for Clinical Research and Health Policy Studies, Tufts Medical Center, Boston, MA. 6Department of Pulmonary and Critical Care Medicine, Lahey Hospital and Medical Center, Burlington, MA.
authors.forename: Georgios D
authors.initials: GD
authors.lastname: Kitsios
authors.forename: Issa J
authors.initials: IJ
authors.lastname: Dahabreh
authors.forename: Sean
authors.initials: S
authors.lastname: Callahan
authors.forename: Jessica K
authors.initials: JK
authors.lastname: Paulus
authors.forename: Anthony C
authors.initials: AC
authors.lastname: Campagna
authors.forename: James M
authors.initials: JM
authors.lastname: Dargin
mesh_terms: D003422:Critical Care; D016638:Critical Illness / Q000401:mortality*; D006801:Humans; D064887:Observational Studies as Topic / Q000706:statistics & numerical data*; D057216:Propensity Score; D016032:Randomized Controlled Trials as Topic / Q000706:statistics & numerical data*