title: A purely confirmatory replication study of structural brain-behavior correlations.
abstract: A recent 'crisis of confidence' has emerged in the empirical sciences. Several studies have suggested that questionable research practices (QRPs) such as optional stopping and selective publication may be relatively widespread. These QRPs can result in a high proportion of false-positive findings, decreasing the reliability and replicability of research output. A potential solution is to register experiments prior to data acquisition and analysis. In this study we attempted to replicate studies that relate brain structure to behavior and cognition. These structural brain-behavior (SBB) correlations occasionally receive much attention in science and in the media. Given the impact of these studies, it is important to investigate their replicability. Here, we attempt to replicate five SBB correlation studies comprising a total of 17 effects. To prevent the impact of QRPs we employed a preregistered, purely confirmatory replication approach. For all but one of the 17 findings under scrutiny, confirmatory Bayesian hypothesis tests indicated evidence in favor of the null hypothesis ranging from anecdotal (Bayes factor < 3) to strong (Bayes factor > 10). In several studies, effect size estimates were substantially lower than in the original studies. To our knowledge, this is the first multi-study confirmatory replication of SBB correlations. With this study, we hope to encourage other researchers to undertake similar replication attempts.
authors: University of Amsterdam, Amsterdam, The Netherlands. Electronic address: W.E.Boekel@uva.nl.
authors: Wouter
authors: W
authors: Boekel
authors: University of Amsterdam, Amsterdam, The Netherlands.
authors: Eric-Jan
authors: EJ
authors: Wagenmakers
authors: University of Amsterdam, Amsterdam, The Netherlands.
authors: Luam
authors: L
authors: Belay
authors: University of Amsterdam, Amsterdam, The Netherlands.
authors: Josine
authors: J
authors: Verhagen
authors: University of Newcastle, Australia.
authors: Scott
authors: S
authors: Brown
authors: University of Amsterdam, Amsterdam, The Netherlands.
authors: Birte U
authors: BU
authors: Forstmann
mesh_terms: D000293:Adolescent; D001499:Bayes Theorem; D001519:Behavior / Q000502:physiology*; D001921:Brain / Q000033:anatomy & histology*; D003071:Cognition / Q000502:physiology*; D038524:Diffusion Magnetic Resonance Imaging; D005260:Female; D006801:Humans; D008279:Magnetic Resonance Imaging; D008297:Male; D009434:Neural Pathways / Q000033:anatomy & histology*; D009929:Organ Size; D015203:Reproducibility of Results; D055815:Young Adult
keywords: Brain-behavior correlations; Confirmatory; Preregistration; Replication