title: Immediate return preference emerged from a synaptic learning rule for return maximization.
abstract: Animals including human often prefer immediate returns to larger delayed returns. It holds true in the human communications. Standard interpretation of the immediate return preference is that an animal might subjectively discount the value of a delayed reward, and that might choose the larger valued one. The interpretation has been successfully applied to explain behavior of many species including human. However, the description is not necessarily sufficient to apply for interactions of individuals. This study adopts a different approach to seek a possibility that immediate return preference may be reproduced by learning rule to maximize objective outcomes. We show that a synaptic learning rule to achieve the temporal difference (TD) learning for outcome maximization fails the maximization and exhibits immediate return preference if the context is not properly represented as a internal state.
authors.affiliation: Graduate School of Brain Sciences, Tamagawa University, Tokyo, Japan.
authors.forename: Yoshiya
authors.initials: Y
authors.lastname: Yamaguchi
authors.affiliation: Graduate School of Brain Sciences, Tamagawa University, Tokyo, Japan; Brain Science Institute, Tamagawa University, Tokyo, Japan.
authors.forename: Takeshi
authors.initials: T
authors.lastname: Aihara
authors.affiliation: Graduate School of Brain Sciences, Tamagawa University, Tokyo, Japan; Brain Science Institute, Tamagawa University, Tokyo, Japan. Electronic address: sakai@eng.tamagawa.ac.jp.
authors.forename: Yutaka
authors.initials: Y
authors.lastname: Sakai
mesh_terms: D000465:Algorithms; D000818:Animals; D010856:Columbidae; D006801:Humans; D007858:Learning / Q000502:physiology*; D051381:Rats; D012201:Reward; D013569:Synapses / Q000502:physiology*
keywords: Delay discount; Inter-temporal choice; Reinforcement learning; Synaptic plasticity