title: Creating a Reinforcement Learning Controller for Functional Electrical Stimulation of a Human Arm.
abstract: Clinical tests have shown that the dynamics of a human arm, controlled using Functional Electrical Stimulation (FES), can vary significantly between and during trials. In this paper, we study the application of Reinforcement Learning to create a controller that can adapt to these changing dynamics of a human arm. Development and tests were done in simulation using a two-dimensional arm model and Hill-based muscle dynamics. An actor-critic architecture is used with artificial neural networks for both the actor and the critic. We begin by training it using a Proportional Derivative (PD) controller as a supervisor. We then make clinically relevant changes to the dynamics of the arm and test the actor-critic's ability to adapt without supervision in a reasonable number of episodes.
authors.affiliation: Department of Electrical Engineering and Computer Science, Case Western Reserve University.
authors.forename: Philip S
authors.initials: PS
authors.lastname: Thomas
authors.forename: Michael
authors.initials: M
authors.lastname: Branicky
authors.forename: Antonie
authors.initials: A
authors.lastname: van den Bogert
authors.forename: Kathleen
authors.initials: K
authors.lastname: Jagodnik
references.citation: Arch Phys Med Rehabil. 2002 Aug;83(8):1093-9
references.pmid: 12161830
references.citation: Biol Cybern. 2004 Jul;91(1):10-22
references.pmid: 15309543
references.citation: IEEE Trans Rehabil Eng. 1998 Jun;6(2):151-61
references.pmid: 9631322
references.citation: IEEE Trans Rehabil Eng. 1997 Mar;5(1):2-11
references.pmid: 9086380
references.citation: Australas Phys Eng Sci Med. 2007 Dec;30(4):317-23
references.pmid: 18274072
references.citation: J Biomech. 2010 Apr 19;43(6):1086-91
references.pmid: 20097345
references.citation: Neural Comput. 2000 Jan;12(1):219-45
references.pmid: 10636940
references.citation: Eur J Neurol. 2008 May;15(5):437-44
references.pmid: 18394046
references.citation: J Biomech Eng. 2003 Dec;125(6):864-74
references.pmid: 14986412
references.citation: Biol Cybern. 1996 Jul;75(1):73-83
references.pmid: 8765656
references.citation: Spinal Cord. 2008 Apr;46(4):255-74
references.pmid: 17846639
references.citation: J Rehabil Res Dev. 1996 Apr;33(2):158-72
references.pmid: 8724171
references.citation: Muscle Nerve. 2007 May;35(5):562-90
references.pmid: 17299744
references.citation: IEEE Trans Rehabil Eng. 1997 Mar;5(1):12-22
references.pmid: 9086381
references.citation: Med Biol Eng Comput. 2002 May;40(3):282-9
references.pmid: 12195974
references.citation: Arch Phys Med Rehabil. 2001 Oct;82(10):1380-8
references.pmid: 11588741
references.citation: Assist Technol. 1992;4(1):19-30
references.pmid: 10148013
references.citation: IEEE Trans Biomed Eng. 1991 Jan;38(1):17-28
references.pmid: 2026428
references.citation: Annu Rev Biomed Eng. 2005;7:327-60
references.pmid: 16004574
references.citation: J Appl Physiol (1985). 1991 Feb;70(2):803-12
references.pmid: 2022572