title: Reinforcement learning in depression: A review of computational research.
abstract: Despite being considered primarily a mood disorder, major depressive disorder (MDD) is characterized by cognitive and decision making deficits. Recent research has employed computational models of reinforcement learning (RL) to address these deficits. The computational approach has the advantage in making explicit predictions about learning and behavior, specifying the process parameters of RL, differentiating between model-free and model-based RL, and the computational model-based functional magnetic resonance imaging and electroencephalography. With these merits there has been an emerging field of computational psychiatry and here we review specific studies that focused on MDD. Considerable evidence suggests that MDD is associated with impaired brain signals of reward prediction error and expected value ('wanting'), decreased reward sensitivity ('liking') and/or learning (be it model-free or model-based), etc., although the causality remains unclear. These parameters may serve as valuable intermediate phenotypes of MDD, linking general clinical symptoms to underlying molecular dysfunctions. We believe future computational research at clinical, systems, and cellular/molecular/genetic levels will propel us toward a better understanding of the disease.
authors.affiliation: Department of Psychiatry, Hokkaido University Graduate School of Medicine, Sapporo 060-8638, Japan. Electronic address: cchen@med.hokudai.ac.jp.
authors.forename: Chong
authors.initials: C
authors.lastname: Chen
authors.affiliation: Department of Behavioral Science/Center for Experimental Research in Social Sciences, Hokkaido University, Sapporo 060-0810, Japan.
authors.forename: Taiki
authors.initials: T
authors.lastname: Takahashi
authors.affiliation: Department of Psychiatry, Hokkaido University Graduate School of Medicine, Sapporo 060-8638, Japan.
authors.forename: Shin
authors.initials: S
authors.lastname: Nakagawa
authors.affiliation: Department of Psychiatry, Hokkaido University Graduate School of Medicine, Sapporo 060-8638, Japan.
authors.forename: Takeshi
authors.initials: T
authors.lastname: Inoue
authors.affiliation: Department of Psychiatry, Hokkaido University Graduate School of Medicine, Sapporo 060-8638, Japan.
authors.forename: Ichiro
authors.initials: I
authors.lastname: Kusumi
mesh_terms: D000818:Animals; D001921:Brain / Q000473:pathology* / Q000503:physiopathology; D003198:Computer Simulation; D003865:Major Depressive Disorder / Q000473:pathology / Q000503:physiopathology / Q000523:psychology; D006801:Humans; D009042:Motivation; D012054:Reinforcement, Psychology
keywords: Anhedonia; Computational psychiatry; Depression; Dopamine; Incentive salience; Learning rate; Model-based; Model-free; Motivation; Prediction error; Reinforcement learning; Reward sensitivity; Stress; ‘Liking’; ‘Wanting’