title: Bayesian deterministic decision making: a normative account of the operant matching law and heavy-tailed reward history dependency of choices.
abstract: The decision making behaviors of humans and animals adapt and then satisfy an "operant matching law" in certain type of tasks. This was first pointed out by Herrnstein in his foraging experiments on pigeons. The matching law has been one landmark for elucidating the underlying processes of decision making and its learning in the brain. An interesting question is whether decisions are made deterministically or probabilistically. Conventional learning models of the matching law are based on the latter idea; they assume that subjects learn choice probabilities of respective alternatives and decide stochastically with the probabilities. However, it is unknown whether the matching law can be accounted for by a deterministic strategy or not. To answer this question, we propose several deterministic Bayesian decision making models that have certain incorrect beliefs about an environment. We claim that a simple model produces behavior satisfying the matching law in static settings of a foraging task but not in dynamic settings. We found that the model that has a belief that the environment is volatile works well in the dynamic foraging task and exhibits undermatching, which is a slight deviation from the matching law observed in many experiments. This model also demonstrates the double-exponential reward history dependency of a choice and a heavier-tailed run-length distribution, as has recently been reported in experiments on monkeys.
authors.affiliation: Department of Complexity Science and Engineering, Graduate School of Frontier Sciences, The University of Tokyo Kashiwa, Japan.
authors.forename: Hiroshi
authors.initials: H
authors.lastname: Saito
authors.affiliation: Center for Evolutionary Cognitive Sciences, The University of Tokyo Tokyo, Japan ; RIKEN Brain Science Institute Wako, Japan ; Okanoya Emotional Information Project, Exploratory Research for Advanced Technology (ERATO), Japan Science and Technology Agency Wako, Japan.
authors.forename: Kentaro
authors.initials: K
authors.lastname: Katahira
authors.affiliation: RIKEN Brain Science Institute Wako, Japan ; Okanoya Emotional Information Project, Exploratory Research for Advanced Technology (ERATO), Japan Science and Technology Agency Wako, Japan ; Department of Life Sciences, Graduate School of Arts and Sciences, The University of Tokyo Tokyo, Japan.
authors.forename: Kazuo
authors.initials: K
authors.lastname: Okanoya
authors.affiliation: Department of Complexity Science and Engineering, Graduate School of Frontier Sciences, The University of Tokyo Kashiwa, Japan ; RIKEN Brain Science Institute Wako, Japan ; Okanoya Emotional Information Project, Exploratory Research for Advanced Technology (ERATO), Japan Science and Technology Agency Wako, Japan.
authors.forename: Masato
authors.initials: M
authors.lastname: Okada
keywords: Bayesian inference; decision making; dynamic foraging task; heavy-tailed reward history dependency; operant matching law