title: Gaze data reveal distinct choice processes underlying model-based and model-free reinforcement learning.
abstract: Organisms appear to learn and make decisions using different strategies known as model-free and model-based learning; the former is mere reinforcement of previously rewarded actions and the latter is a forward-looking strategy that involves evaluation of action-state transition probabilities. Prior work has used neural data to argue that both model-based and model-free learners implement a value comparison process at trial onset, but model-based learners assign more weight to forward-looking computations. Here using eye-tracking, we report evidence for a different interpretation of prior results: model-based subjects make their choices prior to trial onset. In contrast, model-free subjects tend to ignore model-based aspects of the task and instead seem to treat the decision problem as a simple comparison process between two differentially valued items, consistent with previous work on sequential-sampling models of decision making. These findings illustrate a problem with assuming that experimental subjects make their decisions at the same prescribed time.
authors: Department of Economics, The Ohio State University, 1945 North High Street, 410 Arps Hall, Columbus, Ohio 43210, USA.
authors: Arkady
authors: A
authors: Konovalov
authors: Department of Economics, The Ohio State University, 1945 North High Street, 410 Arps Hall, Columbus, Ohio 43210, USA.|Department of Psychology, The Ohio State University, 1827 Neil Avenue, 200E Lazenby Hall, Columbus, Ohio 43210, USA.
authors: Ian
authors: I
authors: Krajbich
mesh_terms: D002755:Choice Behavior; D003657:Decision Making; D005133:Eye Movements; D005260:Female; D005403:Fixation, Ocular; D006801:Humans; D007858:Learning; D016018:Least-Squares Analysis; D008297:Male; D008959:Models, Neurological; D008960:Models, Psychological; D009415:Nerve Net; D011336:Probability; D012044:Regression Analysis; D012054:Reinforcement, Psychology; D012201:Reward