title: Underpowered samples, false negatives, and unconscious learning.
abstract: The scientific community has witnessed growing concern about the high rate of false positives and unreliable results within the psychological literature, but the harmful impact of false negatives has been largely ignored. False negatives are particularly concerning in research areas where demonstrating the absence of an effect is crucial, such as studies of unconscious or implicit processing. Research on implicit processes seeks evidence of above-chance performance on some implicit behavioral measure at the same time as chance-level performance (that is, a null result) on an explicit measure of awareness. A systematic review of 73 studies of contextual cuing, a popular implicit learning paradigm, involving 181 statistical analyses of awareness tests, reveals how underpowered studies can lead to failure to reject a false null hypothesis. Among the studies that reported sufficient information, the meta-analytic effect size across awareness tests was d z = 0.31 (95 % CI 0.24-0.37), showing that participants' learning in these experiments was conscious. The unusually large number of positive results in this literature cannot be explained by selective publication. Instead, our analyses demonstrate that these tests are typically insensitive and underpowered to detect medium to small, but true, effects in awareness tests. These findings challenge a widespread and theoretically important claim about the extent of unconscious human cognition.
authors.affiliation: Primary Care and Public Health Sciences, King's College London, Capital House, 42 Weston St., London, SE1 3QD, UK. miguel.vadillo@kcl.ac.uk.
authors.forename: Miguel A
authors.initials: MA
authors.lastname: Vadillo
authors.affiliation: Department of Social and Decision Sciences, Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA, BP 208, USA. ekonst@cmu.edu.
authors.forename: Emmanouil
authors.initials: E
authors.lastname: Konstantinidis
authors.affiliation: Division of Psychology and Language Sciences, University College London, 26 Bedford Way, London, WC1H 0AH, UK. d.shanks@ucl.ac.uk.
authors.forename: David R
authors.initials: DR
authors.lastname: Shanks
mesh_terms: D000328:Adult; D001364:Awareness / Q000502:physiology*; D003463:Cues; D003627:Data Interpretation, Statistical; D005260:Female; D006801:Humans; D007858:Learning / Q000502:physiology*; D008297:Male; D015201:Meta-Analysis as Topic; D014473:Unconscious, Psychology
keywords: Contextual cuing; False negatives; Implicit learning; Null hypothesis Significance testing· Statistical power