title: Uncertainty and exploration in a restless bandit problem.
abstract: Decision making in noisy and changing environments requires a fine balance between exploiting knowledge about good courses of action and exploring the environment in order to improve upon this knowledge. We present an experiment on a restless bandit task in which participants made repeated choices between options for which the average rewards changed over time. Comparing a number of computational models of participants' behavior in this task, we find evidence that a substantial number of them balanced exploration and exploitation by considering the probability that an option offers the maximum reward out of all the available options.
authors.affiliation: Experimental Psychology, University College London.
authors.forename: Maarten
authors.initials: M
authors.lastname: Speekenbrink
authors.forename: Emmanouil
authors.initials: E
authors.lastname: Konstantinidis
mesh_terms: D000293:Adolescent; D000328:Adult; D003657:Decision Making; D005106:Exploratory Behavior; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D008962:Models, Theoretical; D012201:Reward; D035501:Uncertainty; D055815:Young Adult
keywords: Dynamic decision making; Exploration-exploitation trade-off; Restless multi-armed bandit task; Uncertainty; Volatility