Cross-validation and hypothesis testing in neuroimaging: An irenic comment on the exchange between Friston and Lindquist et al.
The "ten ironic rules for statistical reviewers" presented by Friston (2012) prompted a rebuttal by Lindquist et al. (2013), which was followed by a rejoinder by Friston (2013). A key issue left unresolved in this discussion is the use of cross-validation to test the significance of predictive analyses. This note discusses the role that cross-validation-based and related hypothesis tests have come to play in modern data analyses, in neuroimaging and other fields. It is shown that such tests need not be suboptimal and can fill otherwise-unmet inferential needs.
Department of Child and Adolescent Psychiatry and Department of Population Health, New York University School of Medicine, New York, NY, USA; Nathan S. Kline Institute for Psychiatric Research, Orangeburg, NY, USA. Electronic address: phil.reiss@nyumc.org.
Philip T
PT
Reiss
D059906:Neuroimaging; D017712:Peer Review, Research / Q000379:methods*; D012107:Research Design; D013223:Statistics as Topic / Q000379:methods*
Brain decoding; Cross-validation; Likelihood ratio test; Neyman–Pearson Lemma; Null hypothesis; Permutation test