title: 'Proactive' use of cue-context congruence for building reinforcement learning's reward function.
abstract: Reinforcement learning is a fundamental form of learning that may be formalized using the Bellman equation. Accordingly an agent determines the state value as the sum of immediate reward and of the discounted value of future states. Thus the value of state is determined by agent related attributes (action set, policy, discount factor) and the agent's knowledge of the environment embodied by the reward function and hidden environmental factors given by the transition probability. The central objective of reinforcement learning is to solve these two functions outside the agent's control either using, or not using a model.
  
  
  
  In the present paper, using the proactive model of reinforcement learning we offer insight on how the brain creates simplified representations of the environment, and how these representations are organized to support the identification of relevant stimuli and action. Furthermore, we identify neurobiological correlates of our model by suggesting that the reward and policy functions, attributes of the Bellman equitation, are built by the orbitofrontal cortex (OFC) and the anterior cingulate cortex (ACC), respectively.
  
  
  
  Based on this we propose that the OFC assesses cue-context congruence to activate the most context frame. Furthermore given the bidirectional neuroanatomical link between the OFC and model-free structures, we suggest that model-based input is incorporated into the reward prediction error (RPE) signal, and conversely RPE signal may be used to update the reward-related information of context frames and the policy underlying action selection in the OFC and ACC, respectively. Furthermore clinical implications for cognitive behavioral interventions are discussed.
authors: Department of Health Systems Management and Quality Management for Health Care, Faculty of Public Health, University of Debrecen, Debrecen, Nagyerdei krt. 98, 4032, Hungary. zsuga.judit@med.unideb.hu.
authors: Judit
authors: J
authors: Zsuga
authors: Department of Health Systems Management and Quality Management for Health Care, Faculty of Public Health, University of Debrecen, Debrecen, Nagyerdei krt. 98, 4032, Hungary.
authors: Klara
authors: K
authors: Biro
authors: Department of Health Systems Management and Quality Management for Health Care, Faculty of Public Health, University of Debrecen, Debrecen, Nagyerdei krt. 98, 4032, Hungary.
authors: Gabor
authors: G
authors: Tajti
authors: Department of Pharmacology, Faculty of Pharmacy, University of Debrecen, Debrecen, Nagyerdei krt. 98, 4032, Hungary.
authors: Magdolna Emma
authors: ME
authors: Szilasi
authors: Department of Health Systems Management and Quality Management for Health Care, Faculty of Public Health, University of Debrecen, Debrecen, Nagyerdei krt. 98, 4032, Hungary.
authors: Csaba
authors: C
authors: Papp
authors: Department of Pharmacology and Pharmacotherapy, Faculty of Medicine, University of Debrecen, Debrecen, Nagyerdei krt. 98, 4032, Hungary.
authors: Bela
authors: B
authors: Juhasz
authors: Department of Pharmacology, Faculty of Pharmacy, University of Debrecen, Debrecen, Nagyerdei krt. 98, 4032, Hungary.
authors: Rudolf
authors: R
authors: Gesztelyi
mesh_terms: D000818:Animals; D001244:Association; D002540:Cerebral Cortex / Q000502:physiology*; D003463:Cues; D006801:Humans; D008959:Models, Neurological; D008960:Models, Psychological; D012201:Reward
keywords: Bellman equation; Cue-context congruence; Model-based reinforcement learning; Policy function; Proactive brain; Reward function