title: A training platform for many-dimensional prosthetic devices using a virtual reality environment.
abstract: Brain machine interfaces (BMIs) have the potential to assist in the rehabilitation of millions of patients worldwide. Despite recent advancements in BMI technology for the restoration of lost motor function, a training environment to restore full control of the anatomical segments of an upper limb extremity has not yet been presented. Here, we develop a virtual upper limb prosthesis with 27 independent dimensions, the anatomical dimensions of the human arm and hand, and deploy the virtual prosthesis as an avatar in a virtual reality environment (VRE) that can be controlled in real-time. The prosthesis avatar accepts kinematic control inputs that can be captured from movements of the arm and hand as well as neural control inputs derived from processed neural signals. We characterize the system performance under kinematic control using a commercially available motion capture system. We also present the performance under kinematic control achieved by two non-human primates (Macaca Mulatta) trained to use the prosthetic avatar to perform reaching and grasping tasks. This is the first virtual prosthetic device that is capable of emulating all the anatomical movements of a healthy upper limb in real-time. Since the system accepts both neural and kinematic inputs for a variety of many-dimensional skeletons, we propose it provides a customizable training platform for the acquisition of many-dimensional neural prosthetic control.
authors.affiliation: Center for Neural Science, New York University, New York, NY 10003, United States.
authors.forename: David
authors.initials: D
authors.lastname: Putrino
authors.affiliation: Center for Neural Science, New York University, New York, NY 10003, United States.
authors.forename: Yan T
authors.initials: YT
authors.lastname: Wong
authors.affiliation: Center for Neural Science, New York University, New York, NY 10003, United States.
authors.forename: Adam
authors.initials: A
authors.lastname: Weiss
authors.affiliation: Center for Neural Science, New York University, New York, NY 10003, United States. Electronic address: bijan@nyu.edu.
authors.forename: Bijan
authors.initials: B
authors.lastname: Pesaran
mesh_terms: D000818:Animals; D001186:Artificial Limbs; D001288:Attention / Q000502:physiology; D001676:Biofeedback, Psychology; D001696:Biomechanical Phenomena; D008253:Macaca mulatta; D008954:Models, Biological; D009039:Motion Perception; D019736:Prostheses and Implants; D034941:Upper Extremity / Q000502:physiology*; D014584:User-Computer Interface
keywords: Brain machine interface; Virtual reality environment
references.citation: Disabil Rehabil. 2009;31(10):846-54
references.pmid: 19191061
references.citation: Nat Neurosci. 2002 Nov;5 Suppl:1085-8
references.pmid: 12403992
references.citation: Cyberpsychol Behav. 2006 Apr;9(2):167-70
references.pmid: 16640472
references.citation: Exp Brain Res. 2006 Jun;172(1):77-84
references.pmid: 16424978
references.citation: Cyberpsychol Behav. 2006 Apr;9(2):148-51
references.pmid: 16640468
references.citation: J Neurosci Methods. 2008 Jan 15;167(1):2-14
references.pmid: 18006069
references.citation: J Integr Neurosci. 2011 Dec;10(4):489-511
references.pmid: 22262537
references.citation: IEEE Trans Biomed Eng. 2004 Jun;51(6):1034-43
references.pmid: 15188875
references.citation: Annu Rev Neurosci. 2009;32:249-66
references.pmid: 19400719
references.citation: Behav Brain Res. 1986 May;20(2):185-201
references.pmid: 3730133
references.citation: Nature. 2012 May 17;485(7398):368-71
references.pmid: 22522928
references.citation: IEEE Trans Neural Syst Rehabil Eng. 2010 Oct;18(5):505-14
references.pmid: 20529757
references.citation: Neuroimage. 2006 Apr 1;30(2):436-43
references.pmid: 16300968
references.citation: Ann Biomed Eng. 2005 Jun;33(6):829-40
references.pmid: 16078622
references.citation: PLoS Biol. 2003 Nov;1(2):E42
references.pmid: 14624244
references.citation: IEEE Trans Biomed Eng. 2010 Jul;57(7):1785-97
references.pmid: 20403781
references.citation: Neuroscience. 1985 Nov;16(3):511-20
references.pmid: 4094689
references.citation: Nature. 2011 Oct 05;479(7372):228-31
references.pmid: 21976021
references.citation: J Rehabil Med. 2009 Nov;41(12):1016-102
references.pmid: 19841835
references.citation: IEEE Trans Neural Syst Rehabil Eng. 2007 Mar;15(1):9-15
references.pmid: 17436870
references.citation: IEEE Trans Biomed Eng. 2013 Mar;60(3):792-802
references.pmid: 22287229
references.citation: J Rehabil Res Dev. 2011;48(6):707-18
references.pmid: 21938657
references.citation: J Neurophysiol. 2011 Aug;106(2):564-75
references.pmid: 21562185
references.citation: Clinics (Sao Paulo). 2011;66 Suppl 1:25-32
references.pmid: 21779720
references.citation: IEEE Trans Neural Syst Rehabil Eng. 2012 Mar;20(2):134-42
references.pmid: 22186964
references.citation: IEEE Trans Neural Syst Rehabil Eng. 2003 Mar;11(1):70-85
references.pmid: 12797728
references.citation: Annu Int Conf IEEE Eng Med Biol Soc. 2008;2008:615-9
references.pmid: 19162731
references.citation: Exp Brain Res. 1992;90(3):599-609
references.pmid: 1426116
references.citation: Stud Health Technol Inform. 2012;181:305-9
references.pmid: 22954877
references.citation: IEEE Trans Neural Syst Rehabil Eng. 2001 Sep;9(3):308-18
references.pmid: 11561668
references.citation: J Neural Eng. 2011 Apr;8(2):025001
references.pmid: 21436536
references.citation: J Med Eng Technol. 2006 Jan-Feb;30(1):2-10
references.pmid: 16393847
references.citation: Nat Rev Neurosci. 2011 Nov 03;12(12):752-62
references.pmid: 22048061
references.citation: Exp Brain Res. 2000 Mar;131(1):101-10
references.pmid: 10759175
references.citation: J Neurophysiol. 2010 Aug;104(2):641-53
references.pmid: 20538774