Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling.
Because learned associations are quickly renewed following extinction, the extinction process must include processes other than unlearning. However, reinforcement learning models, such as the temporal difference reinforcement learning (TDRL) model, treat extinction as an unlearning of associated value and are thus unable to capture renewal. TDRL models are based on the hypothesis that dopamine carries a reward prediction error signal; these models predict reward by driving that reward error to zero. The authors construct a TDRL model that can accommodate extinction and renewal through two simple processes: (a) a TDRL process that learns the value of situation-action pairs and (b) a situation recognition process that categorizes the observed cues into situations. This model has implications for dysfunctional states, including relapse after addiction and problem gambling.
Redish
A David
AD
Department of Neuroscience, University of Minnesota.
Jensen
Steve
S
Graduate Program in Computer Science, University of Minnesota.
Johnson
Adam
A
Graduate Program in Neuroscience, University of Minnesota.
Kurth-Nelson
Zeb
Z
Graduate Program in Neuroscience, University of Minnesota.
D000818:Animals; D001245:Association Learning; D003463:Cues; D003657:Decision Making; D005108:Extinction, Psychological; D005715:Gambling / Q000523:psychology*; D006801:Humans; D011939:Mental Recall; D015233:Models, Statistical; D009042:Motivation; D011337:Probability Learning; D012008:Recurrence; D012055:Reinforcement Schedule; D019966:Substance-Related Disorders / Q000523:psychology*