title: Toward a reliable gaze-independent hybrid BCI combining visual and natural auditory stimuli.
abstract: BACKGROUND
  Brain computer interfaces (BCIs) are one of the last communication options for patients in the locked-in state (LIS). For complete LIS patients, interfaces must be gaze-independent due to their eye impairment. However, unimodal gaze-independent approaches typically present levels of performance substantially lower than gaze-dependent approaches. The combination of multimodal stimuli has been pointed as a viable way to increase users' performance.
  
  
  METHODS
  A hybrid visual and auditory (HVA) P300-based BCI combining simultaneously visual and auditory stimulation is proposed. Auditory stimuli are based on natural meaningful spoken words, increasing stimuli discrimination and decreasing user's mental effort in associating stimuli to the symbols. The visual part of the interface is covertly controlled ensuring gaze-independency.
  
  
  RESULTS
  Four conditions were experimentally tested by 10 healthy participants: visual overt (VO), visual covert (VC), auditory (AU) and covert HVA. Average online accuracy for the hybrid approach was 85.3%, which is more than 32% over VC and AU approaches. Questionnaires' results indicate that the HVA approach was the less demanding gaze-independent interface. Interestingly, the P300 grand average for HVA approach coincides with an almost perfect sum of P300 evoked separately by VC and AU tasks.
  
  
  METHODS
  The proposed HVA-BCI is the first solution simultaneously embedding natural spoken words and visual words to provide a communication lexicon. Online accuracy and task demand of the approach compare favorably with state-of-the-art.
  
  
  CONCLUSIONS
  The proposed approach shows that the simultaneous combination of visual covert control and auditory modalities can effectively improve the performance of gaze-independent BCIs.
authors.affiliation: Institute of Systems and Robotics, University of Coimbra, Coimbra, Portugal. Electronic address: sbarbosa@isr.uc.pt.
authors.forename: Sara
authors.initials: S
authors.lastname: Barbosa
authors.affiliation: Institute of Systems and Robotics, University of Coimbra, Coimbra, Portugal; Department of Engineering, Polytechnic Institute of Tomar, Tomar, Portugal. Electronic address: gpires@isr.uc.pt.
authors.forename: Gabriel
authors.initials: G
authors.lastname: Pires
authors.affiliation: Institute of Systems and Robotics, University of Coimbra, Coimbra, Portugal; Department of Electrical and Computer Engineering, University of Coimbra, Coimbra, Portugal. Electronic address: urbano@isr.uc.pt.
authors.forename: Urbano
authors.initials: U
authors.lastname: Nunes
mesh_terms: D000161:Acoustic Stimulation; D000328:Adult; D001307:Auditory Perception / Q000502:physiology*; D001921:Brain / Q000502:physiology*; D062207:Brain-Computer Interfaces; D002138:Calibration; D018913:Event-Related Potentials, P300 / Q000502:physiology*; D005133:Eye Movements; D005240:Feasibility Studies; D005260:Female; D006801:Humans; D008297:Male; D010775:Photic Stimulation; D011782:Quadriplegia / Q000503:physiopathology / Q000628:therapy; D014796:Visual Perception / Q000502:physiology*; D055815:Young Adult
keywords: BCI; Covert control; Gaze-independent; Hybrid visual-auditory; Meaningful natural spoken words; P300