title: Statistical evaluation of forecasts.
abstract: Reliable forecasts of extreme but rare events, such as earthquakes, financial crashes, and epileptic seizures, would render interventions and precautions possible. Therefore, forecasting methods have been developed which intend to raise an alarm if an extreme event is about to occur. In order to statistically validate the performance of a prediction system, it must be compared to the performance of a random predictor, which raises alarms independent of the events. Such a random predictor can be obtained by bootstrapping or analytically. We propose an analytic statistical framework which, in contrast to conventional methods, allows for validating independently the sensitivity and specificity of a forecasting method. Moreover, our method accounts for the periods during which an event has to remain absent or occur after a respective forecast.
authors.affiliation: Department of Neuropediatrics and Muscular Disease, University Medical Center of Freiburg, Freiburg, Germany and Freiburg Center for Data Analysis and Modeling (FDM), University of Freiburg, Freiburg, Germany and Institute for Physics, University of Freiburg, Freiburg, Germany and Center for Neural Engineering, Pennsylvania State University, State College, Pennsylvania 16801, USA and Department of Engineering Science and Mechanics, Pennsylvania State University, State College, Pennsylvania 16801, USA.
authors.forename: Malenka
authors.initials: M
authors.lastname: Mader
authors.affiliation: Freiburg Center for Data Analysis and Modeling (FDM), University of Freiburg, Freiburg, Germany and Institute for Physics, University of Freiburg, Freiburg, Germany and Center for Neural Engineering, Pennsylvania State University, State College, Pennsylvania 16801, USA.
authors.forename: Wolfgang
authors.initials: W
authors.lastname: Mader
authors.affiliation: Center for Neural Engineering, Pennsylvania State University, State College, Pennsylvania 16801, USA and Department of Engineering Science and Mechanics, Pennsylvania State University, State College, Pennsylvania 16801, USA and Department of Neurosurgery, Pennsylvania State University, Hershey, Pennsylvania 17033, USA and Department of Biomedical Engineering, Pennsylvania State University, State College, Pennsylvania 16801, USA.
authors.forename: Bruce J
authors.initials: BJ
authors.lastname: Gluckman
authors.affiliation: Freiburg Center for Data Analysis and Modeling (FDM), University of Freiburg, Freiburg, Germany and Institute for Physics, University of Freiburg, Freiburg, Germany and BIOSS, Center for Biological Signalling Studies, University of Freiburg, Freiburg, Germany.
authors.forename: Jens
authors.initials: J
authors.lastname: Timmer
authors.affiliation: Freiburg Center for Data Analysis and Modeling (FDM), University of Freiburg, Freiburg, Germany and Institute for Complex Systems and Mathematical Biology, University of Aberdeen, Aberdeen, UK.
authors.forename: Björn
authors.initials: B
authors.lastname: Schelter
mesh_terms: D003198:Computer Simulation; D005069:Evaluation Studies as Topic; D005544:Forecasting / Q000379:methods*; D015233:Models, Statistical; D012680:Sensitivity and Specificity