title: A study of the existing problems of estimating the information transfer rate in online brain-computer interfaces.
abstract: OBJECTIVE
  Today, the brain-computer interface (BCI) community lacks a standard method to evaluate an online BCI's performance. Even the most commonly used metric, the information transfer rate (ITR), is often reported differently, even incorrectly, in many papers, which is not conducive to BCI research. This paper aims to point out many of the existing problems and give some suggestions and methods to overcome these problems.
  
  
  METHODS
  First, the preconditions inherent in ITR calculation based on Wolpaw's definition are summarized and several incorrect ITR calculations, which go against the preconditions, are indicated. Then, the issues affecting ITR estimation during the test of online BCI systems are discussed in detail. Finally, a task-oriented online BCI test platform was proposed, which may help BCI evaluations in real-world applications.
  
  
  RESULTS
  The guidelines for ITR calculation in online BCIs testing are proposed. The platform executed in the Beijing BCI Competition 2010 shows that it can be used as a common way to compare the online performances (including the ITR) of existing BCI paradigms.
  
  
  CONCLUSIONS
  The proposed guidelines and task-oriented test platform may reduce the uncertainty and artifacts of online BCI performance evaluation; they provide a relatively objective way to compare different BCI's performances in real-world BCI applications, which is a forward step toward developing standards for BCI performance evaluation.
authors: Department of Biomedical Engineering, Tsinghua University, Beijing, People's Republic of China.
authors: Peng
authors: P
authors: Yuan
authors: Xiaorong
authors: X
authors: Gao
authors: Brendan
authors: B
authors: Allison
authors: Yijun
authors: Y
authors: Wang
authors: Guangyu
authors: G
authors: Bin
authors: Shangkai
authors: S
authors: Gao
mesh_terms: D000465:Algorithms; D016477:Artifacts; D062207:Brain-Computer Interfaces; D003143:Communication Devices for People with Disabilities; D001330:Electronic Data Processing; D004867:Equipment Design; D018913:Event-Related Potentials, P300; D005073:Evoked Potentials, Somatosensory; D017408:Guidelines as Topic; D006801:Humans; D009068:Movement / Q000502:physiology; D009862:Online Systems