title: The value of including observational studies in systematic reviews was unclear: a descriptive study.
abstract: OBJECTIVE
  To evaluate (1) how often observational studies are included in comparative effectiveness reviews (CERs); (2) the rationale for including observational studies; (3) how data from observational studies are appraised, analyzed, and graded; and (4) the impact of observational studies on strength of evidence (SOE) and conclusions.
  
  
  METHODS
  Descriptive study of 23 CERs published through the Effective Health Care Program of the U.S. Agency for Healthcare Research and Quality.
  
  
  RESULTS
  Authors searched for observational studies in 20 CERs, of which 18 included a median of 11 (interquartile range, 2-31) studies. Sixteen CERs incorporated the observational studies in their SOE assessments. Seventy-eight comparisons from 12 CERs included evidence from both trials and observational studies; observational studies had an impact on SOE and conclusions for 19 (24%) comparisons. There was diversity across the CERs regarding decisions to include observational studies; study designs considered; and approaches used to appraise, synthesize, and grade SOE.
  
  
  CONCLUSIONS
  Reporting and methods guidance are needed to ensure clarity and consistency in how observational studies are incorporated in CERs. It was not always clear that observational studies added value in light of the additional resources needed to search for, select, appraise, and analyze such studies.
authors.affiliation: Department of Pediatrics, Alberta Research Centre for Health Evidence and the University of Alberta Evidence-Based Practice Center, University of Alberta, 4-472 ECHA, 11405-87 Avenue, Edmonton, Alberta, Canada T5G 1C9.
authors.forename: Jennifer
authors.initials: J
authors.lastname: Seida
authors.affiliation: Department of Pediatrics, Alberta Research Centre for Health Evidence and the University of Alberta Evidence-Based Practice Center, University of Alberta, 4-472 ECHA, 11405-87 Avenue, Edmonton, Alberta, Canada T5G 1C9.
authors.forename: Donna M
authors.initials: DM
authors.lastname: Dryden
authors.affiliation: Department of Pediatrics, Alberta Research Centre for Health Evidence and the University of Alberta Evidence-Based Practice Center, University of Alberta, 4-472 ECHA, 11405-87 Avenue, Edmonton, Alberta, Canada T5G 1C9. Electronic address: hartling@ualberta.ca.
authors.forename: Lisa
authors.initials: L
authors.lastname: Hartling
mesh_terms: D057186:Comparative Effectiveness Research / Q000379:methods*; D006801:Humans; D064887:Observational Studies as Topic; D011643:Publishing / Q000592:standards* / Q000706:statistics & numerical data; D012196:Review Literature as Topic; D014481:United States; D016270:United States Agency for Healthcare Research and Quality
keywords: Comparative effectiveness; GRADE; Observational studies; Quality of evidence; Strength of evidence; Systematic reviews