title: Empirical Evidence of Study Design Biases in Randomized Trials: Systematic Review of Meta-Epidemiological Studies.
abstract: OBJECTIVE
  To synthesise evidence on the average bias and heterogeneity associated with reported methodological features of randomized trials.
  
  
  METHODS
  Systematic review of meta-epidemiological studies.
  
  
  METHODS
  We retrieved eligible studies included in a recent AHRQ-EPC review on this topic (latest search September 2012), and searched Ovid MEDLINE and Ovid EMBASE for studies indexed from Jan 2012-May 2015. Data were extracted by one author and verified by another. We combined estimates of average bias (e.g. ratio of odds ratios (ROR) or difference in standardised mean differences (dSMD)) in meta-analyses using the random-effects model. Analyses were stratified by type of outcome ("mortality" versus "other objective" versus "subjective"). Direction of effect was standardised so that ROR < 1 and dSMD < 0 denotes a larger intervention effect estimate in trials with an inadequate or unclear (versus adequate) characteristic.
  
  
  RESULTS
  We included 24 studies. The available evidence suggests that intervention effect estimates may be exaggerated in trials with inadequate/unclear (versus adequate) sequence generation (ROR 0.93, 95% CI 0.86 to 0.99; 7 studies) and allocation concealment (ROR 0.90, 95% CI 0.84 to 0.97; 7 studies). For these characteristics, the average bias appeared to be larger in trials of subjective outcomes compared with other objective outcomes. Also, intervention effects for subjective outcomes appear to be exaggerated in trials with lack of/unclear blinding of participants (versus blinding) (dSMD -0.37, 95% CI -0.77 to 0.04; 2 studies), lack of/unclear blinding of outcome assessors (ROR 0.64, 95% CI 0.43 to 0.96; 1 study) and lack of/unclear double blinding (ROR 0.77, 95% CI 0.61 to 0.93; 1 study). The influence of other characteristics (e.g. unblinded trial personnel, attrition) is unclear.
  
  
  CONCLUSIONS
  Certain characteristics of randomized trials may exaggerate intervention effect estimates. The average bias appears to be greatest in trials of subjective outcomes. More research on several characteristics, particularly attrition and selective reporting, is needed.
authors.affiliation: School of Social and Community Medicine, University of Bristol, Bristol, United Kingdom.|School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia.
authors.forename: Matthew J
authors.initials: MJ
authors.lastname: Page
authors.affiliation: School of Social and Community Medicine, University of Bristol, Bristol, United Kingdom.
authors.forename: Julian P T
authors.initials: JP
authors.lastname: Higgins
authors.affiliation: School of Social and Community Medicine, University of Bristol, Bristol, United Kingdom.
authors.forename: Gemma
authors.initials: G
authors.lastname: Clayton
authors.affiliation: School of Social and Community Medicine, University of Bristol, Bristol, United Kingdom.
authors.forename: Jonathan A C
authors.initials: JA
authors.lastname: Sterne
authors.affiliation: Center for Evidence-Based Medicine, University of Southern Denmark & Odense University Hospital, Odense, Denmark.
authors.forename: Asbjørn
authors.initials: A
authors.lastname: Hróbjartsson
authors.affiliation: School of Social and Community Medicine, University of Bristol, Bristol, United Kingdom.|The National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care West (NIHR CLAHRC West) at University Hospitals Bristol NHS Foundation Trust, Bristol, United Kingdom.
authors.forename: Jelena
authors.initials: J
authors.lastname: Savović
mesh_terms: D015982:Bias; D016021:Epidemiologic Studies; D006801:Humans; D016032:Randomized Controlled Trials as Topic / Q000379:methods*