title: Blending of brain-machine interface and vision-guided autonomous robotics improves neuroprosthetic arm performance during grasping.
abstract: BACKGROUND
  Recent studies have shown that brain-machine interfaces (BMIs) offer great potential for restoring upper limb function. However, grasping objects is a complicated task and the signals extracted from the brain may not always be capable of driving these movements reliably. Vision-guided robotic assistance is one possible way to improve BMI performance. We describe a method of shared control where the user controls a prosthetic arm using a BMI and receives assistance with positioning the hand when it approaches an object.
  
  
  METHODS
  Two human subjects with tetraplegia used a robotic arm to complete object transport tasks with and without shared control. The shared control system was designed to provide a balance between BMI-derived intention and computer assistance. An autonomous robotic grasping system identified and tracked objects and defined stable grasp positions for these objects. The system identified when the user intended to interact with an object based on the BMI-controlled movements of the robotic arm. Using shared control, BMI controlled movements and autonomous grasping commands were blended to ensure secure grasps.
  
  
  RESULTS
  Both subjects were more successful on object transfer tasks when using shared control compared to BMI control alone. Movements made using shared control were more accurate, more efficient, and less difficult. One participant attempted a task with multiple objects and successfully lifted one of two closely spaced objects in 92 % of trials, demonstrating the potential for users to accurately execute their intention while using shared control.
  
  
  CONCLUSIONS
  Integration of BMI control with vision-guided robotic assistance led to improved performance on object transfer tasks. Providing assistance while maintaining generalizability will make BMI systems more attractive to potential users.
  
  
  BACKGROUND
  NCT01364480 and NCT01894802 .
authors.affiliation: Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA.|Center for the Neural Basis of Cognition, Pittsburgh, PA, USA.
authors.forename: John E
authors.initials: JE
authors.lastname: Downey
authors.affiliation: Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA.|Department of Physical Medicine and Rehabilitation, University of Pittsburgh, Pittsburgh, PA, USA.
authors.forename: Jeffrey M
authors.initials: JM
authors.lastname: Weiss
authors.affiliation: Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA.
authors.forename: Katharina
authors.initials: K
authors.lastname: Muelling
authors.affiliation: Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA.
authors.forename: Arun
authors.initials: A
authors.lastname: Venkatraman
authors.affiliation: Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA.
authors.forename: Jean-Sebastien
authors.initials: JS
authors.lastname: Valois
authors.affiliation: Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA.
authors.forename: Martial
authors.initials: M
authors.lastname: Hebert
authors.affiliation: Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA.
authors.forename: J Andrew
authors.initials: JA
authors.lastname: Bagnell
authors.affiliation: Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA.|Center for the Neural Basis of Cognition, Pittsburgh, PA, USA.|Department of Neurobiology, University of Pittsburgh, Pittsburgh, PA, USA.
authors.forename: Andrew B
authors.initials: AB
authors.lastname: Schwartz
authors.affiliation: Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA. collinger@pitt.edu.|Center for the Neural Basis of Cognition, Pittsburgh, PA, USA. collinger@pitt.edu.|Department of Physical Medicine and Rehabilitation, University of Pittsburgh, Pittsburgh, PA, USA. collinger@pitt.edu.|VA Pittsburgh Healthcare System, Pittsburgh, PA, USA. collinger@pitt.edu.|University of Pittsburgh, 3520 5th Avenue, Suite 300, Pittsburgh, PA, 15213, USA. collinger@pitt.edu.
authors.forename: Jennifer L
authors.identifier: http://orcid.org/0000-0002-4517-5395
authors.initials: JL
authors.lastname: Collinger
mesh_terms: D000328:Adult; D001921:Brain / Q000503:physiopathology; D062207:Brain-Computer Interfaces; D005260:Female; D006225:Hand / Q000503:physiopathology; D018737:Hand Strength; D006801:Humans; D008297:Male; D008875:Middle Aged; D009068:Movement; D000066530:Neurological Rehabilitation / Q000295:instrumentation*; D011782:Quadriplegia / Q000503:physiopathology; D012371:Robotics / Q000379:methods*; D034941:Upper Extremity / Q000503:physiopathology
keywords: Assistive technology; Brain-computer interface; Brain-machine interface; Neuroprosthetic; Shared mode control