title: The use of Bayesian networks to assess the quality of evidence from research synthesis: 1.
abstract: BACKGROUND
  The grades of recommendation, assessment, development and evaluation (GRADE) approach is widely implemented in systematic reviews, health technology assessment and guideline development organisations throughout the world. A key advantage to this approach is that it aids transparency regarding judgments on the quality of evidence. However, the intricacies of making judgments about research methodology and evidence make the GRADE system complex and challenging to apply without training.
  
  
  METHODS
  We have developed a semi-automated quality assessment tool (SAQAT) l based on GRADE. This is informed by responses by reviewers to checklist questions regarding characteristics that may lead to unreliability. These responses are then entered into the Bayesian network to ascertain the probabilities of risk of bias, inconsistency, indirectness, imprecision and publication bias conditional on review characteristics. The model then combines these probabilities to provide a probability for each of the GRADE overall quality categories. We tested the model using a range of plausible scenarios that guideline developers or review authors could encounter.
  
  
  RESULTS
  Overall, the model reproduced GRADE judgements for a range of scenarios. Potential advantages over standard assessment are use of explicit and consistent weightings for different review characteristics, forcing consideration of important but sometimes neglected characteristics and principled downgrading where small but important probabilities of downgrading are accrued across domains.
  
  
  CONCLUSIONS
  Bayesian networks have considerable potential for use as tools to assess the validity of research evidence. The key strength of such networks lies in the provision of a statistically coherent method for combining probabilities across a complex framework based on both belief and evidence. In addition to providing tools for less experienced users to implement reliability assessment, the potential for sensitivity analyses and automation may be beneficial for application and the methodological development of reliability tools.
authors.affiliation: Centre for Reviews and Dissemination, University of York, York, United Kingdom.
authors.forename: Gavin B
authors.initials: GB
authors.lastname: Stewart
authors.affiliation: Centre for Reviews and Dissemination, University of York, York, United Kingdom; School of Social and Community Medicine, University of Bristol, Bristol, United Kingdom.
authors.forename: Julian P T
authors.initials: JP
authors.lastname: Higgins
authors.affiliation: Department of Clinical Epidemiology & Biostatistics, McMaster University Health Sciences Centre, Hamilton, ON, Canada.
authors.forename: Holger
authors.initials: H
authors.lastname: Schünemann
authors.affiliation: Centre for Reviews and Dissemination, University of York, York, United Kingdom.
authors.forename: Nick
authors.initials: N
authors.lastname: Meader
mesh_terms: D001499:Bayes Theorem; D057189:Checklist; D016021:Epidemiologic Studies; D055317:Evidence-Based Practice / Q000379:methods*; D006801:Humans; D015201:Meta-Analysis as Topic; D015233:Models, Statistical; D016032:Randomized Controlled Trials as Topic; D012107:Research Design; D013673:Technology Assessment, Biomedical / Q000379:methods*
references.citation: J Clin Epidemiol. 2011 Apr;64(4):395-400
references.pmid: 21194891
references.citation: J Clin Epidemiol. 2013 Feb;66(2):151-7
references.pmid: 22542023
references.citation: Cochrane Database Syst Rev. 2013 Feb 28;(2):CD003409
references.pmid: 23450540
references.citation: J Clin Epidemiol. 2011 Dec;64(12):1294-302
references.pmid: 21803546
references.citation: Cochrane Database Syst Rev. 2012 May 16;(5):CD003053
references.pmid: 22592687
references.citation: Cochrane Database Syst Rev. 2012 Nov 14;11:CD006391
references.pmid: 23152236
references.citation: J Clin Epidemiol. 2013 Feb;66(2):140-50
references.pmid: 22863410
references.citation: J Clin Epidemiol. 2011 Dec;64(12):1277-82
references.pmid: 21802904
references.citation: PLoS One. 2012;7(4):e34697
references.pmid: 22496843
references.citation: Cochrane Database Syst Rev. 2013 Jan 31;(1):CD000313
references.pmid: 23440778
references.citation: J Clin Epidemiol. 2013 Jul;66(7):719-25
references.pmid: 23312392
references.citation: J Clin Epidemiol. 2011 Apr;64(4):401-6
references.pmid: 21208779
references.citation: PLoS Med. 2005 Aug;2(8):e124
references.pmid: 16060722
references.citation: Syst Rev. 2014 Jul 24;3:82
references.pmid: 25056145
references.citation: PLoS Med. 2009 Sep;6(9):e1000094
references.pmid: 19753107
references.citation: J Clin Epidemiol. 2011 Apr;64(4):380-2
references.pmid: 21185693
references.citation: J Clin Epidemiol. 2013 Feb;66(2):173-83
references.pmid: 23116689
references.citation: J Clin Epidemiol. 2011 Dec;64(12):1311-6
references.pmid: 21802902
references.citation: Syst Rev. 2012 Feb 21;1:15
references.pmid: 22588052
references.citation: J Clin Epidemiol. 2011 Dec;64(12):1283-93
references.pmid: 21839614
references.citation: Genet Med. 2012 Jul;14(7):663-9
references.pmid: 22481134
references.citation: J Clin Epidemiol. 2013 Feb;66(2):158-72
references.pmid: 22609141
references.citation: J Clin Epidemiol. 2011 Dec;64(12):1303-10
references.pmid: 21802903
references.citation: J Clin Epidemiol. 2011 Apr;64(4):383-94
references.pmid: 21195583
references.citation: J Clin Epidemiol. 2011 Apr;64(4):407-15
references.pmid: 21247734
references.citation: PLoS One. 2015 Dec 30;10(12):e0123511
references.pmid: 26716874