title: Neural Circuits Trained with Standard Reinforcement Learning Can Accumulate Probabilistic Information during Decision Making.
abstract: Much experimental evidence suggests that during decision making, neural circuits accumulate evidence supporting alternative options. A computational model well describing this accumulation for choices between two options assumes that the brain integrates the log ratios of the likelihoods of the sensory inputs given the two options. Several models have been proposed for how neural circuits can learn these log-likelihood ratios from experience, but all of these models introduced novel and specially dedicated synaptic plasticity rules. Here we show that for a certain wide class of tasks, the log-likelihood ratios are approximately linearly proportional to the expected rewards for selecting actions. Therefore, a simple model based on standard reinforcement learning rules is able to estimate the log-likelihood ratios from experience and on each trial accumulate the log-likelihood ratios associated with presented stimuli while selecting an action. The simulations of the model replicate experimental data on both behavior and neural activity in tasks requiring accumulation of probabilistic cues. Our results suggest that there is no need for the brain to support dedicated plasticity rules, as the standard mechanisms proposed to describe reinforcement learning can enable the neural circuits to perform efficient probabilistic inference.
authors.affiliation: Medical Research Council Brain Network Dynamics Unit, University of Oxford, Oxford, OX1 3QT, U.K., and Institute of Pharmacy and Molecular Biotechnology, University of Heidelberg, D-69120 Heidelberg, Germany n.kurzawa@stud.uni-heidelberg.de.
authors.forename: Nils
authors.initials: N
authors.lastname: Kurzawa
authors.affiliation: Department of Experimental Psychology, University of Oxford, Oxford OX1 3UD, U.K. christopher.summerfield@psy.ox.ac.uk.
authors.forename: Christopher
authors.initials: C
authors.lastname: Summerfield
authors.affiliation: Medical Research Council Brain Network Dynamics Unit, University of Oxford, Oxford OX1 3UD, U.K., and Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford OX1 3UD, U.K. rafal.bogacz@ndcn.ox.ac.uk.
authors.forename: Rafal
authors.initials: R
authors.lastname: Bogacz