title: The successor representation and temporal context.
abstract: The successor representation was introduced into reinforcement learning by Dayan ( 1993 ) as a means of facilitating generalization between states with similar successors. Although reinforcement learning in general has been used extensively as a model of psychological and neural processes, the psychological validity of the successor representation has yet to be explored. An interesting possibility is that the successor representation can be used not only for reinforcement learning but for episodic learning as well. Our main contribution is to show that a variant of the temporal context model (TCM; Howard & Kahana,  2002 ), an influential model of episodic memory, can be understood as directly estimating the successor representation using the temporal difference learning algorithm (Sutton & Barto,  1998 ). This insight leads to a generalization of TCM and new experimental predictions. In addition to casting a new normative light on TCM, this equivalence suggests a previously unexplored point of contact between different learning systems.
authors.affiliation: Department of Psychology and Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08540, USA. sjgershm@princeton.edu
authors.forename: Samuel J
authors.initials: SJ
authors.lastname: Gershman
authors.forename: Christopher D
authors.initials: CD
authors.lastname: Moore
authors.forename: Michael T
authors.initials: MT
authors.lastname: Todd
authors.forename: Kenneth A
authors.initials: KA
authors.lastname: Norman
authors.forename: Per B
authors.initials: PB
authors.lastname: Sederberg
mesh_terms: D000465:Algorithms; D007858:Learning / Q000502:physiology*; D008960:Models, Psychological; D015203:Reproducibility of Results; D013997:Time Factors