title: Incorporation of assessments of risk of bias of primary studies in systematic reviews of randomised trials: a cross-sectional study.
abstract: OBJECTIVE
  We examined how assessments of risk of bias of primary studies are carried out and incorporated into the statistical analysis and overall findings of a systematic review.
  
  
  METHODS
  A cross-sectional review.
  
  
  METHODS
  We assessed 200 systematic reviews of randomised trials published between January and March 2012; Cochrane (n=100), non-Cochrane (Database of Reviews of Effects) (n=100).
  
  
  RESULTS
  Our primary outcome was a descriptive analysis of how assessments of risk of bias are carried out, the methods used, and the extent to which such assessments were incorporated into the statistical analysis and overall review findings.
  
  
  RESULTS
  While Cochrane reviews routinely reported the method of risk of bias assessment and presented their results either in text or table format, 20% of non-Cochrane reviews failed to report the method used and 39% did not present the assessment results. Where it was possible to evaluate the individual results of the risk of bias assessment (n=154), 75% (n=116/154) of reviews had ≥1 trial at high risk of bias; the median proportion of trials per review at high risk of bias was 50% (IQR 31% to 89%). Despite this, only 56% (n=65/116) incorporated the risk of bias assessment into the interpretation of the results in the abstract and 41% (n=47/116) (49%; n=40/81 Cochrane and 20%; n=7/35 non-Cochrane) incorporated the risk of bias assessment into the interpretation of the conclusions. Of the 83% (n=166/200) systematic reviews which included a meta-analysis, only 11% (n=19/166) incorporated the risk of bias assessment into the statistical analysis.
  
  
  CONCLUSIONS
  Cochrane reviews were more likely than non-Cochrane reviews to report how risk of bias assessments of primary studies were carried out; however, both frequently failed to take such assessments into account in the statistical analysis and conclusions of the systematic review.
authors: INSERM, U738, Paris, France.
authors: Sally
authors: S
authors: Hopewell
authors: Isabelle
authors: I
authors: Boutron
authors: Douglas G
authors: DG
authors: Altman
authors: Philippe
authors: P
authors: Ravaud
keywords: Epidemiology; General Medicine (see Internal Medicine); Statistics & Research Methods
references: JAMA. 1999 Sep 15;282(11):1054-60
references: 10493204
references: Int J Epidemiol. 2007 Aug;36(4):847-57
references: 17517809
references: PLoS Med. 2010 Sep 21;7(9):e1000326
references: 20877712
references: Ann Intern Med. 2012 Sep 18;157(6):429-38
references: 22945832
references: J Clin Epidemiol. 2012 Mar;65(3):236-8
references: 22264767
references: BMJ. 2011 Oct 18;343:d5928
references: 22008217
references: BMC Med Res Methodol. 2008 Apr 21;8:22
references: 18426565
references: J Clin Epidemiol. 2011 Apr;64(4):407-15
references: 21247734
references: PLoS Med. 2007 Mar 27;4(3):e78
references: 17388659
references: BMJ. 2005 May 7;330(7499):1053
references: 15817526
references: BMJ. 2008 Apr 26;336(7650):924-6
references: 18436948
references: Control Clin Trials. 1996 Feb;17(1):1-12
references: 8721797
references: Ann Intern Med. 2009 Aug 18;151(4):W65-94
references: 19622512
references: BMJ. 2001 Jul 7;323(7303):42-6
references: 11440947
references: JAMA. 1998 Jul 15;280(3):278-80
references: 9676681
references: BMJ. 2009 Jul 21;339:b2535
references: 19622551
references: PLoS One. 2008 Aug 28;3(8):e3081
references: 18769481