A valid and reliable belief elicitation method for Bayesian priors.
OBJECTIVE
Bayesian inference has the advantage of formally incorporating prior beliefs about the effect of an intervention into analyses of treatment effect through the use of prior probability distributions or "priors." Multiple methods to elicit beliefs from experts for inclusion in a Bayesian study have been used; however, the measurement properties of these methods have been infrequently evaluated. The objectives of this study were to evaluate the feasibility, validity, and reliability of a belief elicitation method for Bayesian priors.


METHODS
A single-center, cross-sectional study using a sample of academic specialists who treat pulmonary hypertension patients was conducted to test the feasibility, face and construct validity, and reliability of a belief elicitation method. Using this method, participants expressed the probability of 3-year survival with and without warfarin. Applying adhesive dots or "chips," each representing 5% probability, in "bins" on a line, participants expressed their uncertainty and weight of belief about the effect of warfarin on 3-year survival.


RESULTS
Of the 12 participants, 11 (92%) reported that the belief elicitation method had face validity, 10 (83%) found the questions clear, and 11 (92%) found the response option easy to use. The median time to completion was 10 minutes (5-15 minutes). Internal validity testing found moderate agreement (weighted kappa=0.54-0.57). The intraclass correlation coefficient for test-retest reliability was 0.93.


CONCLUSIONS
This method of belief elicitation for Bayesian priors is feasible, valid, and reliable. It can be considered for application in Bayesian clinical studies.
Johnson
Sindhu R
SR
Division of Rheumatology, Department of Medicine, Toronto Western Hospital, University Health Network, Toronto, Ontario M5T 2S8, Canada. Sindhu.Johnson@uhn.on.ca
Tomlinson
George A
GA
Hawker
Gillian A
GA
Granton
John T
JT
Grosbein
Haddas A
HA
Feldman
Brian M
BM
D000925:Anticoagulants / Q000627:therapeutic use; D001499:Bayes Theorem; D015982:Bias; D002170:Canada / Q000453:epidemiology; D003430:Cross-Sectional Studies; D006801:Humans; D006976:Hypertension, Pulmonary / Q000188:drug therapy* / Q000401:mortality; D011336:Probability; D015203:Reproducibility of Results; D011795:Surveys and Questionnaires; D016896:Treatment Outcome; D014859:Warfarin / Q000627:therapeutic use
D000925:Anticoagulants; D014859:Warfarin