title: A power fallacy.
abstract: The power fallacy refers to the misconception that what holds on average -across an ensemble of hypothetical experiments- also holds for each case individually. According to the fallacy, high-power experiments always yield more informative data than do low-power experiments. Here we expose the fallacy with concrete examples, demonstrating that a particular outcome from a high-power experiment can be completely uninformative, whereas a particular outcome from a low-power experiment can be highly informative. Although power is useful in planning an experiment, it is less useful-and sometimes even misleading-for making inferences from observed data. To make inferences from data, we recommend the use of likelihood ratios or Bayes factors, which are the extension of likelihood ratios beyond point hypotheses. These methods of inference do not average over hypothetical replications of an experiment, but instead condition on the data that have actually been observed. In this way, likelihood ratios and Bayes factors rationally quantify the evidence that a particular data set provides for or against the null or any other hypothesis.
authors.affiliation: Department of Psychology, University of Amsterdam, Weesperplein 4, 1018 XA, Amsterdam, The Netherlands. ej.wagenmakers@gmail.com.
authors.forename: Eric-Jan
authors.initials: EJ
authors.lastname: Wagenmakers
authors.affiliation: Department of Psychology, University of Amsterdam, Weesperplein 4, 1018 XA, Amsterdam, The Netherlands.
authors.forename: Josine
authors.initials: J
authors.lastname: Verhagen
authors.affiliation: Department of Psychology, University of Amsterdam, Weesperplein 4, 1018 XA, Amsterdam, The Netherlands.
authors.forename: Alexander
authors.initials: A
authors.lastname: Ly
authors.affiliation: Department of Psychology, University of Amsterdam, Weesperplein 4, 1018 XA, Amsterdam, The Netherlands.
authors.forename: Marjan
authors.initials: M
authors.lastname: Bakker
authors.affiliation: University of California Irvine, Irvine, CA, USA.
authors.forename: Michael D
authors.initials: MD
authors.lastname: Lee
authors.affiliation: Department of Psychology, University of Amsterdam, Weesperplein 4, 1018 XA, Amsterdam, The Netherlands.
authors.forename: Dora
authors.initials: D
authors.lastname: Matzke
authors.affiliation: University of Missouri, Columbia, MO, USA.
authors.forename: Jeffrey N
authors.initials: JN
authors.lastname: Rouder
authors.affiliation: University of Groningen, Groningen, Netherlands.
authors.forename: Richard D
authors.initials: RD
authors.lastname: Morey
mesh_terms: D001499:Bayes Theorem; D006801:Humans; D018401:Sample Size; D013223:Statistics as Topic
keywords: Bayes factor; Hypothesis test; Likelihood ratio; Statistical evidence