Policy relevance of Bayesian statistics overestimated?
OBJECTIVE
The observed posterior probability distributions regarding the benefits of surgery for otitis media with effusion (OME) with expected probability distributions, using Bayes' theorem are compared.


METHODS
Postal questionnaires were used to assess prior and posterior probability distributions among ear-nose-throat (ENT) surgeons in the Netherlands.


RESULTS
In their prior probability estimates, ENT surgeons were quite optimistic with respect to the effectiveness of tube insertion in the treatment of OME. The trial showed no meaningful benefit of tubes on hearing and language development. Posterior probabilities calculated on the basis of prior probability estimates and trial results differed widely from those, elicited empirically 1 year after completion of the trial and dissemination of the results.


CONCLUSIONS
ENT surgeons did not adjust their opinion about the benefits of surgical treatment of glue ears to the extent that they should have done according to Bayes' theorem. Users of the results of Bayesian analyses, notably policy-makers, should realize that Bayes' theorem is prescriptive and not necessarily descriptively correct. Health policy decisions should not be based on the untested assumption that health-care professionals use new evidence to adjust their subjective beliefs in a Bayesian manner.
van der Wilt
Gert Jan
GJ
Department of Medical Technology Assessment, University Medical Centre Nijmegen, The Netherlands. G.vanderwilt@mta.umcn.nl
Rovers
Maroeska
M
Straatman
Huub
H
van der Bij
Sjoukje
S
van den Broek
Paul
P
Zielhuis
Gerhard
G
D001291:Attitude of Health Personnel; D001499:Bayes Theorem; D002648:Child; D006291:Health Policy; D006801:Humans; D033181:Information Dissemination; D010034:Otitis Media with Effusion / Q000601:surgery*; D010820:Physicians; D016032:Randomized Controlled Trials as Topic; D011795:Surveys and Questionnaires; D013673:Technology Assessment, Biomedical / Q000706:statistics & numerical data