title: Structure learning in human sequential decision-making.
abstract: Studies of sequential decision-making in humans frequently find suboptimal performance relative to an ideal actor that has perfect knowledge of the model of how rewards and events are generated in the environment. Rather than being suboptimal, we argue that the learning problem humans face is more complex, in that it also involves learning the structure of reward generation in the environment. We formulate the problem of structure learning in sequential decision tasks using Bayesian reinforcement learning, and show that learning the generative model for rewards qualitatively changes the behavior of an optimal learning agent. To test whether people exhibit structure learning, we performed experiments involving a mixture of one-armed and two-armed bandit reward models, where structure learning produces many of the qualitative behaviors deemed suboptimal in previous studies. Our results demonstrate humans can perform structure learning in a near-optimal manner.
authors.affiliation: Department of Computer Science and Engineering, University of Minnesota, Minneapolis, Minnesota, United States of America. acuna002@umn.edu
authors.forename: Daniel E
authors.initials: DE
authors.lastname: Acuña
authors.forename: Paul
authors.initials: P
authors.lastname: Schrater
mesh_terms: D000465:Algorithms; D001499:Bayes Theorem; D003657:Decision Making / Q000502:physiology*; D006801:Humans; D007858:Learning / Q000502:physiology*; D008962:Models, Theoretical; D012201:Reward; D013647:Task Performance and Analysis