title: Homeostatic reinforcement learning for integrating reward collection and physiological stability.
abstract: Efficient regulation of internal homeostasis and defending it against perturbations requires adaptive behavioral strategies. However, the computational principles mediating the interaction between homeostatic and associative learning processes remain undefined. Here we use a definition of primary rewards, as outcomes fulfilling physiological needs, to build a normative theory showing how learning motivated behaviors may be modulated by internal states. Within this framework, we mathematically prove that seeking rewards is equivalent to the fundamental objective of physiological stability, defining the notion of physiological rationality of behavior. We further suggest a formal basis for temporal discounting of rewards by showing that discounting motivates animals to follow the shortest path in the space of physiological variables toward the desired setpoint. We also explain how animals learn to act predictively to preclude prospective homeostatic challenges, and several other behavioral patterns. Finally, we suggest a computational role for interaction between hypothalamus and the brain reward system.
authors.affiliation: Group for Neural Theory, INSERM U960, Departément des Etudes Cognitives, Ecole Normale Supérieure, PSL Research University, Paris, France.
authors.forename: Mehdi
authors.initials: M
authors.lastname: Keramati
authors.affiliation: Group for Neural Theory, INSERM U960, Departément des Etudes Cognitives, Ecole Normale Supérieure, PSL Research University, Paris, France.
authors.forename: Boris
authors.initials: B
authors.lastname: Gutkin
mesh_terms: D000818:Animals; D001519:Behavior; D003198:Computer Simulation; D003213:Conditioning, Psychological; D005108:Extinction, Psychological; D006706:Homeostasis / Q000502:physiology*; D007858:Learning; D008959:Models, Neurological; D012054:Reinforcement, Psychology; D012201:Reward; D012528:Satiety Response
keywords: anticipatory responding; cortico-basal ganglia; homeostatic regulation; hypothalamus; neuroscience; none; reinforcement learning; temporal discounting