title: Crowdsourcing reproducible seizure forecasting in human and canine epilepsy.
abstract: SEE MORMANN AND ANDRZEJAK DOI101093/BRAIN/AWW091 FOR A SCIENTIFIC COMMENTARY ON THIS ARTICLE  : Accurate forecasting of epileptic seizures has the potential to transform clinical epilepsy care. However, progress toward reliable seizure forecasting has been hampered by lack of open access to long duration recordings with an adequate number of seizures for investigators to rigorously compare algorithms and results. A seizure forecasting competition was conducted on kaggle.com using open access chronic ambulatory intracranial electroencephalography from five canines with naturally occurring epilepsy and two humans undergoing prolonged wide bandwidth intracranial electroencephalographic monitoring. Data were provided to participants as 10-min interictal and preictal clips, with approximately half of the 60 GB data bundle labelled (interictal/preictal) for algorithm training and half unlabelled for evaluation. The contestants developed custom algorithms and uploaded their classifications (interictal/preictal) for the unknown testing data, and a randomly selected 40% of data segments were scored and results broadcasted on a public leader board. The contest ran from August to November 2014, and 654 participants submitted 17 856 classifications of the unlabelled test data. The top performing entry scored 0.84 area under the classification curve. Following the contest, additional held-out unlabelled data clips were provided to the top 10 participants and they submitted classifications for the new unseen data. The resulting area under the classification curves were well above chance forecasting, but did show a mean 6.54 ± 2.45% (min, max: 0.30, 20.2) decline in performance. The kaggle.com model using open access data and algorithms generated reproducible research that advanced seizure forecasting. The overall performance from multiple contestants on unseen data was better than a random predictor, and demonstrates the feasibility of seizure forecasting in canine and human epilepsy.media-1vid110.1093/brain/aww045_video_abstractaww045_video_abstract.
authors.affiliation: Mayo Systems Electrophysiology Laboratory, Departments of Neurology and Biomedical Engineering, Mayo Clinic, Rochester, MN 55905, USA Brinkmann.Benjamin@mayo.edu.
authors.forename: Benjamin H
authors.initials: BH
authors.lastname: Brinkmann
authors.affiliation: University of Pennsylvania, Penn Center for Neuroengineering and Therapeutics, Philadelphia, PA, USA.
authors.forename: Joost
authors.initials: J
authors.lastname: Wagenaar
authors.affiliation: AiLive Inc, Sunnyvale, CA, USA.
authors.forename: Drew
authors.initials: D
authors.lastname: Abbot
authors.affiliation: AiLive Inc, Sunnyvale, CA, USA.
authors.forename: Phillip
authors.initials: P
authors.lastname: Adkins
authors.affiliation: University of Queensland, Centre for Advanced Imaging, Queensland, Australia.
authors.forename: Simone C
authors.initials: SC
authors.lastname: Bosshard
authors.affiliation: University of Queensland, Centre for Advanced Imaging, Queensland, Australia.
authors.forename: Min
authors.initials: M
authors.lastname: Chen
authors.affiliation: University of Queensland, Centre for Advanced Imaging, Queensland, Australia.
authors.forename: Quang M
authors.initials: QM
authors.lastname: Tieng
authors.affiliation: Hemedics Inc, Boston, MA, USA.
authors.forename: Jialune
authors.initials: J
authors.lastname: He
authors.affiliation: CEU Cardenal Herrera University, Valencia, Spain.
authors.forename: F J
authors.initials: FJ
authors.lastname: Muñoz-Almaraz
authors.affiliation: CEU Cardenal Herrera University, Valencia, Spain.
authors.forename: Paloma
authors.initials: P
authors.lastname: Botella-Rocamora
authors.affiliation: CEU Cardenal Herrera University, Valencia, Spain.
authors.forename: Juan
authors.initials: J
authors.lastname: Pardo
authors.affiliation: CEU Cardenal Herrera University, Valencia, Spain.
authors.forename: Francisco
authors.initials: F
authors.lastname: Zamora-Martinez
authors.affiliation: Sydney, Australia.
authors.forename: Michael
authors.initials: M
authors.lastname: Hills
authors.affiliation: New York, NY, USA.
authors.forename: Wei
authors.initials: W
authors.lastname: Wu
authors.affiliation: Ghent University, Ghent, Belgium.
authors.forename: Iryna
authors.initials: I
authors.lastname: Korshunova
authors.affiliation: Kaggle, Inc. New York NY, USA.
authors.forename: Will
authors.initials: W
authors.lastname: Cukierski
authors.affiliation: University of Pennsylvania, School of Veterinary Medicine Philadelphia, PA, USA.
authors.forename: Charles
authors.initials: C
authors.lastname: Vite
authors.affiliation: University of Minnesota, Veterinary Medical Center, St. Paul, MN, USA.
authors.forename: Edward E
authors.initials: EE
authors.lastname: Patterson
authors.affiliation: University of Pennsylvania, Penn Center for Neuroengineering and Therapeutics, Philadelphia, PA, USA.
authors.forename: Brian
authors.initials: B
authors.lastname: Litt
authors.affiliation: Mayo Systems Electrophysiology Laboratory, Departments of Neurology and Biomedical Engineering, Mayo Clinic, Rochester, MN 55905, USA.
authors.forename: Gregory A
authors.initials: GA
authors.lastname: Worrell
mesh_terms: D000368:Aged; D000465:Algorithms; D000818:Animals; D063045:Crowdsourcing; D004285:Dogs; D042241:Early Diagnosis; D004567:Electrodes, Implanted; D004569:Electroencephalography; D004827:Epilepsy / Q000175:diagnosis*; D005260:Female; D005544:Forecasting / Q000379:methods*; D006801:Humans; D008875:Middle Aged; D008991:Monitoring, Physiologic / Q000379:methods; D012640:Seizures / Q000175:diagnosis*
keywords: epilepsy; experimental models; intracranial EEG; refractory epilepsy