Adaptive learning via selectionism and Bayesianism, Part II: the sequential case.
Animals increase or decrease their future tendency of emitting an action based on whether performing such action has, in the past, resulted in positive or negative reinforcement. An analysis in the companion paper [Zhang, J. (2009). Adaptive learning via selectionism and Bayesianism. Part I: Connection between the two. Neural Networks, 22(3), 220-228] of such selectionist style of learning reveals a resemblance between its ensemble-level dynamics governing the change of action probability and Bayesian learning where evidence (in this case, reward) is distributively applied to all action alternatives. Here, this equivalence is further explored in solving the temporal credit-assignment problem during the learning of an action sequence ("operant chain"). Naturally emerging are the notion of secondary (conditioned) reinforcement predicting the average reward associated with a stimulus, and the notion of actor-critic architecture involving concurrent learning of both action probability and reward prediction. While both are consistent with solutions provided by contemporary reinforcement learning theory (Sutton & Barto, 1998) for optimizing sequential decision-making under stationary Markov environments, we investigate the effect of action learning on reward prediction when both are carried out concurrently in any on-line scheme.
Zhang
Jun
J
Department of Psychology, University of Michigan, 530 Church Street, Ann Arbor 48109-1043, USA. junz@umich.edu
D000223:Adaptation, Psychological / Q000502:physiology*; D000818:Animals; D001499:Bayes Theorem; D003198:Computer Simulation; D003216:Conditioning, Operant / Q000502:physiology; D003657:Decision Making / Q000502:physiology; D006801:Humans; D007858:Learning / Q000502:physiology*; D008390:Markov Chains; D008606:Mental Processes / Q000502:physiology; D009483:Neuropsychological Tests; D011337:Probability Learning; D011597:Psychomotor Performance / Q000502:physiology; D012054:Reinforcement, Psychology