title: Methods to elicit beliefs for Bayesian priors: a systematic review.
abstract: OBJECTIVE
  Bayesian analysis can incorporate clinicians' beliefs about treatment effectiveness into models that estimate treatment effects. Many elicitation methods are available, but it is unclear if any confer advantages based on principles of measurement science. We review belief-elicitation methods for Bayesian analysis and determine if any of them had an incremental value over the others based on its validity, reliability, and responsiveness.
  
  
  METHODS
  A systematic review was performed. MEDLINE, EMBASE, CINAHL, Health and Psychosocial Instruments, Current Index to Statistics, MathSciNet, and Zentralblatt Math were searched using the terms (prior OR prior probability distribution) AND (beliefs OR elicitation) AND (Bayes OR Bayesian). Studies were evaluated on: design, question stem, response options, analysis, consideration of validity, reliability, and responsiveness.
  
  
  RESULTS
  We identified 33 studies describing methods for elicitation in a Bayesian context. Elicitation occurred in cross-sectional studies (n=30, 89%), to derive point estimates with individual-level variation (n=19; 58%). Although 64% (n=21) considered validity, 24% (n=8) reliability, 12% (n=4) responsiveness of the elicitation methods, only 12% (n=4) formally tested validity, 6% (n=2) tested reliability, and none tested responsiveness.
  
  
  CONCLUSIONS
  We have summarized methods of belief elicitation for Bayesian priors. The validity, reliability, and responsiveness of elicitation methods have been infrequently evaluated. Until comparative studies are performed, strategies to reduce the effects of bias on the elicitation should be used.
authors.affiliation: Division of Rheumatology, Department of Medicine, University Health Network, Toronto, Ontario M5T 2S8, Canada. Sindhu.Johnson@uhn.on.ca
authors.forename: Sindhu R
authors.initials: SR
authors.lastname: Johnson
authors.forename: George A
authors.initials: GA
authors.lastname: Tomlinson
authors.forename: Gillian A
authors.initials: GA
authors.lastname: Hawker
authors.forename: John T
authors.initials: JT
authors.lastname: Granton
authors.forename: Brian M
authors.initials: BM
authors.lastname: Feldman
mesh_terms: D001499:Bayes Theorem; D015982:Bias; D002986:Clinical Trials as Topic; D003657:Decision Making; D006801:Humans; D011336:Probability; D015203:Reproducibility of Results; D012108:Research Personnel / Q000523:psychology*; D016896:Treatment Outcome