title: How do systematic reviews incorporate risk of bias assessments into the synthesis of evidence? A methodological study.
abstract: BACKGROUND
  Systematic reviews (SRs) are expected to critically appraise included studies and privilege those at lowest risk of bias (RoB) in the synthesis. This study examines if and how critical appraisals inform the synthesis and interpretation of evidence in SRs.
  
  
  METHODS
  All SRs published in March-May 2012 in 14 high-ranked medical journals and a sample from the Cochrane library were systematically assessed by two reviewers to determine if and how: critical appraisal was conducted; RoB was summarised at study, domain and review levels; and RoB appraisals informed the synthesis process.
  
  
  RESULTS
  Of the 59 SRs studied, all except six (90%) conducted a critical appraisal of the included studies, with most using or adapting existing tools. Almost half of the SRs reported critical appraisal in a manner that did not allow readers to determine which studies included in a review were most robust. RoB assessments were not incorporated into synthesis in one-third (20) of the SRs, with their consideration more likely when reviews focused on randomised controlled trials. Common methods for incorporating critical appraisals into the synthesis process were sensitivity analysis, narrative discussion and exclusion of studies at high RoB. Nearly half of the reviews which investigated multiple outcomes and carried out study-level RoB summaries did not consider the potential for RoB to vary across outcomes.
  
  
  CONCLUSIONS
  The conclusions of the SRs, published in major journals, are frequently uninformed by the critical appraisal process, even when conducted. This may be particularly problematic for SRs of public health topics that often draw on diverse study designs.
authors.affiliation: Evaluation of Social Interventions Programme, MRC/CSO Social and Public Health Sciences Unit, University of Glasgow, Glasgow, UK.
authors.forename: Srinivasa Vittal
authors.initials: SV
authors.lastname: Katikireddi
authors.affiliation: Social and Environmental Health Research Department, London School of Hygiene and Tropical Medicine, London, UK.
authors.forename: Matt
authors.initials: M
authors.lastname: Egan
authors.affiliation: Social and Environmental Health Research Department, London School of Hygiene and Tropical Medicine, London, UK.
authors.forename: Mark
authors.initials: M
authors.lastname: Petticrew
mesh_terms: D015982:Bias; D019317:Evidence-Based Medicine / Q000379:methods / Q000592:standards*; D006801:Humans; D012107:Research Design; D012196:Review Literature as Topic; D018570:Risk Assessment
keywords: EPIDEMIOLOGY; Epidemiological methods; PUBLIC HEALTH; SYSTEMATIC REVIEWS