title: Statistically comparing EEG/MEG waveforms through successive significant univariate tests: how bad can it be?
abstract: When making statistical comparisons, the temporal dimension of the EEG signal introduces problems. Guthrie and Buchwald (1991) proposed a formally correct statistical approach that deals with these problems: comparing waveforms by counting the number of successive significant univariate tests and then contrasting this number to a well-chosen critical value. However, in the literature, this method is often used inappropriately. Using real EEG data and Monte Carlo simulations, we examined the problems associated with the incorrect use of this approach under circumstances often encountered in the literature. Our results show inflated false-positive or false-negative rates depending on parameters of the data, including filtering. Our findings suggest that most applications of this method result in an inappropriate familywise error rate control. Solutions and alternative methods are discussed.
authors.affiliation: Department of Psychology and the Helen Wills Neuroscience Institute, University of California at Berkeley, Berkeley, California, USA; Radboud University Nijmegen, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands.
authors.forename: Vitória
authors.initials: V
authors.lastname: Piai
authors.forename: Kristoffer
authors.initials: K
authors.lastname: Dahlslätt
authors.forename: Eric
authors.initials: E
authors.lastname: Maris
mesh_terms: D000465:Algorithms; D001921:Brain / Q000502:physiology*; D001931:Brain Mapping / Q000379:methods; D003198:Computer Simulation; D004569:Electroencephalography / Q000379:methods*; D006801:Humans; D015225:Magnetoencephalography / Q000379:methods*; D015233:Models, Statistical; D013223:Statistics as Topic
keywords: Autocorrelation; EEG/ERP; Multiple comparisons problem; Statistics; Successive univariate tests