title: A symbiotic brain-machine interface through value-based decision making.
abstract: BACKGROUND
  In the development of Brain Machine Interfaces (BMIs), there is a great need to enable users to interact with changing environments during the activities of daily life. It is expected that the number and scope of the learning tasks encountered during interaction with the environment as well as the pattern of brain activity will vary over time. These conditions, in addition to neural reorganization, pose a challenge to decoding neural commands for BMIs. We have developed a new BMI framework in which a computational agent symbiotically decoded users' intended actions by utilizing both motor commands and goal information directly from the brain through a continuous Perception-Action-Reward Cycle (PARC).
  
  
  METHODS
  The control architecture designed was based on Actor-Critic learning, which is a PARC-based reinforcement learning method. Our neurophysiology studies in rat models suggested that Nucleus Accumbens (NAcc) contained a rich representation of goal information in terms of predicting the probability of earning reward and it could be translated into an evaluative feedback for adaptation of the decoder with high precision. Simulated neural control experiments showed that the system was able to maintain high performance in decoding neural motor commands during novel tasks or in the presence of reorganization in the neural input. We then implanted a dual micro-wire array in the primary motor cortex (M1) and the NAcc of rat brain and implemented a full closed-loop system in which robot actions were decoded from the single unit activity in M1 based on an evaluative feedback that was estimated from NAcc.
  
  
  CONCLUSIONS
  Our results suggest that adapting the BMI decoder with an evaluative feedback that is directly extracted from the brain is a possible solution to the problem of operating BMIs in changing environments with dynamic neural signals. During closed-loop control, the agent was able to solve a reaching task by capturing the action and reward interdependency in the brain.
authors.affiliation: Department of Biomedical Engineering, University of Miami, Coral Gables, Florida, United States of America. b.mahmoudi@gmail.com
authors.forename: Babak
authors.initials: B
authors.lastname: Mahmoudi
authors.forename: Justin C
authors.initials: JC
authors.lastname: Sanchez
mesh_terms: D000465:Algorithms; D000818:Animals; D001522:Behavior, Animal; D003198:Computer Simulation; D003657:Decision Making; D008328:Man-Machine Systems; D009714:Nucleus Accumbens / Q000502:physiology; D051381:Rats; D013238:Stereotaxic Techniques; D014584:User-Computer Interface