Real-time decision fusion for multimodal neural prosthetic devices.
BACKGROUND
The field of neural prosthetics aims to develop prosthetic limbs with a brain-computer interface (BCI) through which neural activity is decoded into movements. A natural extension of current research is the incorporation of neural activity from multiple modalities to more accurately estimate the user's intent. The challenge remains how to appropriately combine this information in real-time for a neural prosthetic device.


RESULTS
Here we propose a framework based on decision fusion, i.e., fusing predictions from several single-modality decoders to produce a more accurate device state estimate. We examine two algorithms for continuous variable decision fusion: the Kalman filter and artificial neural networks (ANNs). Using simulated cortical neural spike signals, we implemented several successful individual neural decoding algorithms, and tested the capabilities of each fusion method in the context of decoding 2-dimensional endpoint trajectories of a neural prosthetic arm. Extensively testing these methods on random trajectories, we find that on average both the Kalman filter and ANNs successfully fuse the individual decoder estimates to produce more accurate predictions.


CONCLUSIONS
Our results reveal that a fusion-based approach has the potential to improve prediction accuracy over individual decoders of varying quality, and we hope that this work will encourage multimodal neural prosthetics experiments in the future.
White
James Robert
JR
Applied Mathematics and Scientific Computation Program, University of Maryland-College Park, College Park, Maryland, United States of America. whitej@umd.edu
Levy
Todd
T
Bishop
William
W
Beaty
James D
JD
D000465:Algorithms; D005121:Extremities; D006801:Humans; D008328:Man-Machine Systems; D008959:Models, Neurological; D015233:Models, Statistical; D009044:Motor Cortex; D009068:Movement; D016571:Neural Networks, Computer; D016011:Normal Distribution; D019736:Prostheses and Implants; D011474:Prosthesis Design; D015203:Reproducibility of Results; D012984:Software; D014584:User-Computer Interface