Hypothesis testing and theory evaluation at the boundaries: surprising insights from Bayes's theorem.
Because the probability of obtaining an experimental finding given that the null hypothesis is true [p(F\H0)] is not the same as the probability that the null hypothesis is true given a finding [p(H0\F)], calculating the former probability does not justify conclusions about the latter one. As the standard null-hypothesis significance-testing procedure does just that, it is logically invalid (J. Cohen, 1994). Theoretically, Bayes's theorem yields p(H0\F), but in practice, researchers rarely know the correct values for 2 of the variables in the theorem. Nevertheless, by considering a wide range of possible values for the unknown variables, it is possible to calculate a range of theoretical values for p(H0\F) and to draw conclusions about both hypothesis testing and theory evaluation.
Trafimow
David
D
Department of Psychology, New Mexico State University, Las Cruces 88003-8001, USA. trafimow@crl.nmsu.edu
D001499:Bayes Theorem; D006801:Humans; D015233:Models, Statistical; D011584:Psychology / Q000379:methods; D012107:Research Design