title: Reliable gains? Evidence for substantially underpowered designs in studies of working memory training transfer to fluid intelligence.
abstract: In recent years, cognitive scientists and commercial interests (e.g., Fit Brains, Lumosity) have focused research attention and financial resources on cognitive tasks, especially working memory tasks, to explore and exploit possible transfer effects to general cognitive abilities, such as fluid intelligence. The increased research attention has produced mixed findings, as well as contention about the disposition of the evidence base. To address this contention, Au et al. (2014) recently conducted a meta-analysis of extant controlled experimental studies of n-back task training transfer effects on measures of fluid intelligence in healthy adults; the results of which showed a small training transfer effect. Using several approaches, the current review evaluated and re-analyzed the meta-analytic data for the presence of two different forms of small-study effects: (1) publication bias in the presence of low power and; (2) low power in the absence of publication bias. The results of these approaches showed no evidence of selection bias in the working memory training literature, but did show evidence of small-study effects related to low power in the absence of publication bias. While the effect size estimate identified by Au et al. (2014) provided the most precise estimate to date, it should be interpreted in the context of a uniformly low-powered base of evidence. The present work concludes with a brief set of considerations for assessing the adequacy of a body of research findings for the application of meta-analytic techniques.
authors: Department of Psychology, Wayne State University Detroit, MI, USA.
authors: Tim
authors: T
authors: Bogg
authors: Department of Psychology, Wayne State University Detroit, MI, USA.
authors: Leanne
authors: L
authors: Lasecki
keywords: cognitive training intervention; fluid intelligence; meta-analysis; small-study effects; statistical power; transfer effects
references: Am Psychol. 2012 Feb-Mar;67(2):130-59
references: 22233090
references: Psychol Methods. 2001 Sep;6(3):203-17
references: 11570228
references: Nat Rev Neurosci. 2013 May;14 (5):365-76
references: 23571845
references: Biostatistics. 2011 Jan;12(1):122-42
references: 20656692
references: PLoS One. 2013 May 22;8(5):e63614
references: 23717453
references: N Engl J Med. 2008 Jan 17;358(3):252-60
references: 18199864
references: Psychol Rev. 2007 Jan;114(1):104-32
references: 17227183
references: Front Psychol. 2014 Jul 30;5:823
references: 25126083
references: BMJ. 2009 Aug 07;339:b2981
references: 19666685
references: PLoS Med. 2005 Aug;2(8):e124
references: 16060722
references: Psychon Bull Rev. 2015 Apr;22(2):366-77
references: 25102926
references: Mem Cognit. 2014 Apr;42(3):464-80
references: 24081919
references: Clin Trials. 2007;4(3):245-53
references: 17715249
references: BMJ. 2003 Sep 6;327(7414):557-60
references: 12958120
references: Proc Natl Acad Sci U S A. 2008 May 13;105(19):6829-33
references: 18443283
references: Res Synth Methods. 2014 Mar;5(1):60-78
references: 26054026
references: J Clin Epidemiol. 2008 Oct;61(10):991-6
references: 18538991
references: J Clin Epidemiol. 2005 Jun;58(6):543-9
references: 15878467
references: J Clin Epidemiol. 2011 Oct;64(10):1060-9
references: 21454050
references: Psychon Bull Rev. 2011 Feb;18(1):46-60
references: 21327348
references: Epidemiology. 2008 Sep;19(5):640-8
references: 18633328