title: Optimizing event-related potential based brain-computer interfaces: a systematic evaluation of dynamic stopping methods.
abstract: OBJECTIVE
  In brain-computer interface (BCI) research, systems based on event-related potentials (ERP) are considered particularly successful and robust. This stems in part from the repeated stimulation which counteracts the low signal-to-noise ratio in electroencephalograms. Repeated stimulation leads to an optimization problem, as more repetitions also cost more time. The optimal number of repetitions thus represents a data-dependent trade-off between the stimulation time and the obtained accuracy. Several methods for dealing with this have been proposed as 'early stopping', 'dynamic stopping' or 'adaptive stimulation'. Despite their high potential for BCI systems at the patient's bedside, those methods are typically ignored in current BCI literature. The goal of the current study is to assess the benefit of these methods.
  
  
  METHODS
  This study assesses for the first time the existing methods on a common benchmark of both artificially generated data and real BCI data of 83 BCI sessions, allowing for a direct comparison between these methods in the context of text entry.
  
  
  RESULTS
  The results clearly show the beneficial effect on the online performance of a BCI system, if the trade-off between the number of stimulus repetitions and accuracy is optimized. All assessed methods work very well for data of good subjects, and worse for data of low-performing subjects. Most methods, however, are robust in the sense that they do not reduce the performance below the baseline of a simple no stopping strategy.
  
  
  CONCLUSIONS
  Since all methods can be realized as a module between the BCI and an application, minimal changes are needed to include these methods into existing BCI software architectures. Furthermore, the hyperparameters of most methods depend to a large extend on only a single variable-the discriminability of the training data. For the convenience of BCI practitioners, the present study proposes linear regression coefficients for directly estimating the hyperparameters from the data based on this discriminability. The data that were used in this publication are made publicly available to benchmark future methods.
authors.affiliation: Machine Learning Laboratory, Berlin Institute of Technology, Marchstrasse 23, 10537, Berlin, Germany. schreuder@tu-berlin.de
authors.forename: Martijn
authors.initials: M
authors.lastname: Schreuder
authors.forename: Johannes
authors.initials: J
authors.lastname: Höhne
authors.forename: Benjamin
authors.initials: B
authors.lastname: Blankertz
authors.forename: Stefan
authors.initials: S
authors.lastname: Haufe
authors.forename: Thorsten
authors.initials: T
authors.lastname: Dickhaus
authors.forename: Michael
authors.initials: M
authors.lastname: Tangermann
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D001921:Brain / Q000502:physiology*; D001931:Brain Mapping / Q000379:methods*; D062207:Brain-Computer Interfaces; D004569:Electroencephalography / Q000379:methods*; D005071:Evoked Potentials / Q000502:physiology*; D006801:Humans; D010363:Pattern Recognition, Automated / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity