Classic or Bayesian research design and analysis. Does it make a difference?
OBJECTIVE
The role of classical and Bayesian statistical approaches remains in dispute in health services research and policy. The goal of this study was to determine if results differ when both analytic techniques are used with the same data set.


METHODS
We searched MEDLINE and related databases for English-language articles published January 1, 1978 through August 31, 1999. We combined Bayesian and classical statistics search terms and their variants with randomized control trials (RCTs) and meta-analyses.


RESULTS
Searches found 18 studies in 14 publications that met all review criteria--nine RCTs, eight meta-analyses, and one epidemiologic estimate. Statistical analyses using both methods agreed in five RCTs, four meta-analyses, and for the epidemiologic estimates. For four RCTs where results disagreed, classical analysis found the experimental intervention was efficacious compared with the control, and Bayesian reanalysis concluded the intervention was not proven efficacious. Classical meta-analyses of the four studies where results disagreed concluded the experimental intervention was not better than the control; Bayesian reanalysis concluded it was efficacious.


CONCLUSIONS
Classical and Bayesian methods in this review exhibited important divergence of results. Disagreement on many fundamental beliefs between classical and Bayesian statistics means continuing debate. One way to resolve this debate is for proponents of each technique to decide together the circumstances for use of each method and analytic framework. If the experts do not agree on the methodologic requirements, other decision makers likely will force their own views.
Bloom
Bernard S
BS
University of Pennsylvania, USA.
de Pouvourville
Nathalie
N
Libert
Simon
S
D001499:Bayes Theorem; D003661:Decision Support Techniques; D015276:Health Care Rationing; D006291:Health Policy; D006302:Health Services Research / Q000379:methods*; D006801:Humans; D015201:Meta-Analysis as Topic; D016032:Randomized Controlled Trials as Topic; D012107:Research Design; D013673:Technology Assessment, Biomedical / Q000379:methods*; D014481:United States