title: Researchers' Intuitions About Power in Psychological Research.
abstract: Many psychology studies are statistically underpowered. In part, this may be because many researchers rely on intuition, rules of thumb, and prior practice (along with practical considerations) to determine the number of subjects to test. In Study 1, we surveyed 291 published research psychologists and found large discrepancies between their reports of their preferred amount of power and the actual power of their studies (calculated from their reported typical cell size, typical effect size, and acceptable alpha). Furthermore, in Study 2, 89% of the 214 respondents overestimated the power of specific research designs with a small expected effect size, and 95% underestimated the sample size needed to obtain .80 power for detecting a small effect. Neither researchers' experience nor their knowledge predicted the bias in their self-reported power intuitions. Because many respondents reported that they based their sample sizes on rules of thumb or common practice in the field, we recommend that researchers conduct and report formal power analyses for their studies.
authors.affiliation: Department of Methodology and Statistics, Tilburg School of Social and Behavioral Sciences, Tilburg University M.Bakker_1@uvt.nl.
authors.forename: Marjan
authors.initials: M
authors.lastname: Bakker
authors.affiliation: Department of Methodology and Statistics, Tilburg School of Social and Behavioral Sciences, Tilburg University.
authors.forename: Chris H J
authors.initials: CH
authors.lastname: Hartgerink
authors.affiliation: Department of Methodology and Statistics, Tilburg School of Social and Behavioral Sciences, Tilburg University.
authors.forename: Jelte M
authors.initials: JM
authors.lastname: Wicherts
authors.affiliation: Department of Psychology, Psychological Methods, University of Amsterdam.
authors.forename: Han L J
authors.initials: HL
authors.lastname: van der Maas
mesh_terms: D006801:Humans; D019545:Intuition / Q000502:physiology*; D019359:Knowledge; D011584:Psychology; D012106:Research; D012107:Research Design; D012108:Research Personnel / Q000523:psychology*; D018401:Sample Size; D057566:Self Report; D011795:Surveys and Questionnaires
keywords: effect size; methodology; open data; open materials; power; sample size; survey