title: Evaluating the TD model of classical conditioning.
abstract: The temporal-difference (TD) algorithm from reinforcement learning provides a simple method for incrementally learning predictions of upcoming events. Applied to classical conditioning, TD models suppose that animals learn a real-time prediction of the unconditioned stimulus (US) on the basis of all available conditioned stimuli (CSs). In the TD model, similar to other error-correction models, learning is driven by prediction errors--the difference between the change in US prediction and the actual US. With the TD model, however, learning occurs continuously from moment to moment and is not artificially constrained to occur in trials. Accordingly, a key feature of any TD model is the assumption about the representation of a CS on a moment-to-moment basis. Here, we evaluate the performance of the TD model with a heretofore unexplored range of classical conditioning tasks. To do so, we consider three stimulus representations that vary in their degree of temporal generalization and evaluate how the representation influences the performance of the TD model on these conditioning tasks.
authors.affiliation: Princeton Neuroscience Institute and Department of Mechanical & Aerospace Engineering, Princeton University, 3-N-12 Green Hall, Princeton, NJ 08542, USA. eludvig@princeton.edu
authors.forename: Elliot A
authors.initials: EA
authors.lastname: Ludvig
authors.forename: Richard S
authors.initials: RS
authors.lastname: Sutton
authors.forename: E James
authors.initials: EJ
authors.lastname: Kehoe
mesh_terms: D000465:Algorithms; D000818:Animals; D001245:Association Learning; D003198:Computer Simulation / Q000706:statistics & numerical data; D003214:Conditioning, Classical; D008960:Models, Psychological; D012054:Reinforcement, Psychology; D013997:Time Factors