title: Towards autonomous neuroprosthetic control using Hebbian reinforcement learning.
abstract: OBJECTIVE
  Our goal was to design an adaptive neuroprosthetic controller that could learn the mapping from neural states to prosthetic actions and automatically adjust adaptation using only a binary evaluative feedback as a measure of desirability/undesirability of performance.
  
  
  METHODS
  Hebbian reinforcement learning (HRL) in a connectionist network was used for the design of the adaptive controller. The method combines the efficiency of supervised learning with the generality of reinforcement learning. The convergence properties of this approach were studied using both closed-loop control simulations and open-loop simulations that used primate neural data from robot-assisted reaching tasks.
  
  
  RESULTS
  The HRL controller was able to perform classification and regression tasks using its episodic and sequential learning modes, respectively. In our experiments, the HRL controller quickly achieved convergence to an effective control policy, followed by robust performance. The controller also automatically stopped adapting the parameters after converging to a satisfactory control policy. Additionally, when the input neural vector was reorganized, the controller resumed adaptation to maintain performance.
  
  
  CONCLUSIONS
  By estimating an evaluative feedback directly from the user, the HRL control algorithm may provide an efficient method for autonomous adaptation of neuroprosthetic systems. This method may enable the user to teach the controller the desired behavior using only a simple feedback signal.
authors.affiliation: Department of Neurosurgery, Emory University, Atlanta, GA, USA.
authors.forename: Babak
authors.initials: B
authors.lastname: Mahmoudi
authors.forename: Eric A
authors.initials: EA
authors.lastname: Pohlmeyer
authors.forename: Noeline W
authors.initials: NW
authors.lastname: Prins
authors.forename: Shijia
authors.initials: S
authors.lastname: Geng
authors.forename: Justin C
authors.initials: JC
authors.lastname: Sanchez
mesh_terms: D000465:Algorithms; D000818:Animals; D001185:Artificial Intelligence / Q000592:standards*; D002144:Callithrix; D007858:Learning / Q000502:physiology*; D058117:Neural Prostheses / Q000592:standards*; D011897:Random Allocation; D012054:Reinforcement, Psychology