title: Prospective Optimization.
abstract: Human performance approaches that of an ideal observer and optimal actor in some perceptual and motor tasks. These optimal abilities depend on the capacity of the cerebral cortex to store an immense amount of information and to flexibly make rapid decisions. However, behavior only approaches these limits after a long period of learning while the cerebral cortex interacts with the basal ganglia, an ancient part of the vertebrate brain that is responsible for learning sequences of actions directed toward achieving goals. Progress has been made in understanding the algorithms used by the brain during reinforcement learning, which is an online approximation of dynamic programming. Humans also make plans that depend on past experience by simulating different scenarios, which is calledprospective optimization. The same brain structures in the cortex and basal ganglia that are active online during optimal behavior are also active offline during prospective optimization. The emergence of general principles and algorithms for goal-directed behavior has consequences for the development of autonomous devices in engineering applications.
authors.affiliation: Howard Hughes Medical Institute, Salk Institute for Biological Sciences, La Jolla, CA 92037 USA and the Division of Biological Studies, University of California at San Diego, La Jolla, CA 92093 USA terry@salk.edu ).
authors.forename: Terrence J
authors.initials: TJ
authors.lastname: Sejnowski
authors.affiliation: Institute for Neural Computation, University of California at San Diego, La Jolla, CA 92093-0523 USA ( hpoizner@ucsd.edu ).
authors.forename: Howard
authors.initials: H
authors.lastname: Poizner
authors.affiliation: Department of Psychiatry and Human Behavior, University of California at Irvine, Irvine, CA 92697-4292 USA ( glynch@uci.edu ).
authors.forename: Gary
authors.initials: G
authors.lastname: Lynch
authors.affiliation: Systems Neurobiology Laboratories, Salk Institute for University of California at San Diego, La Jolla, CA 92037 USA ( sergei@salk.edu ).
authors.forename: Sergei
authors.initials: S
authors.lastname: Gepshtein
authors.affiliation: Kavli Institute for Brain and Mind, University of California at San Diego, La Jolla, CA 92093-0126 USA ( rgreenspan@ucsd.edu ).
authors.forename: Ralph J
authors.initials: RJ
authors.lastname: Greenspan
keywords: Basal ganglia; cerebral cortex; classical conditioning; dynamic programming; hippocampus; ideal observer; limbic system; optimization; reinforcement learning; temporal-difference learning