title: Statistical validation of event predictors: a comparative study based on the field of seizure prediction.
abstract: The prediction of events is of substantial interest in many research areas. To evaluate the performance of prediction methods, the statistical validation of these methods is of utmost importance. Here, we compare an analytical validation method to numerical approaches that are based on Monte Carlo simulations. The comparison is performed in the field of the prediction of epileptic seizures. In contrast to the analytical validation method, we found that for numerical validation methods insufficient but realistic sample sizes can lead to invalid high rates of false positive conclusions. Hence we outline necessary preconditions for sound statistical tests on above chance predictions.
authors.affiliation: Bernstein Center Freiburg, University of Freiburg, Freiburg, Germany and Freiburg Center for Data Analysis and Modeling, University of Freiburg, Freiburg, Germany. hinnerk.feldwisch@bcf.uni-freiburg.de
authors.forename: Hinnerk
authors.initials: H
authors.lastname: Feldwisch-Drentrup
authors.forename: Andreas
authors.initials: A
authors.lastname: Schulze-Bonhage
authors.forename: Jens
authors.initials: J
authors.lastname: Timmer
authors.forename: Björn
authors.initials: B
authors.lastname: Schelter
mesh_terms: D005544:Forecasting / Q000379:methods*; D015233:Models, Statistical; D009010:Monte Carlo Method; D012640:Seizures