Simple artificial neural networks that match probability and exploit and explore when confronting a multiarmed bandit.
The matching law (Herrnstein 1961) states that response rates become proportional to reinforcement rates; this is related to the empirical phenomenon called probability matching (Vulkan 2000). Here, we show that a simple artificial neural network generates responses consistent with probability matching. This behavior was then used to create an operant procedure for network learning. We use the multiarmed bandit (Gittins 1989), a classic problem of choice behavior, to illustrate that operant training balances exploiting the bandit arm expected to pay off most frequently with exploring other arms. Perceptrons provide a medium for relating results from neural networks, genetic algorithms, animal learning, contingency theory, reinforcement learning, and theories of choice.
Dawson
Michael R W
MR
Department of Psychology, University of Alberta, Edmonton, AB T6G 2P9, Canada. mdawson@ualberta.ca
Dupuis
Brian
B
Spetch
Marcia L
ML
Kelly
Debbie M
DM
D000818:Animals; D001185:Artificial Intelligence; D002755:Choice Behavior; D003198:Computer Simulation; D003216:Conditioning, Operant; D006801:Humans; D016571:Neural Networks, Computer; D011336:Probability; D012055:Reinforcement Schedule; D012054:Reinforcement, Psychology