Surprise beyond prediction error.
Surprise drives learning. Various neural "prediction error" signals are believed to underpin surprise-based reinforcement learning. Here, we report a surprise signal that reflects reinforcement learning but is neither un/signed reward prediction error (RPE) nor un/signed state prediction error (SPE). To exclude these alternatives, we measured surprise responses in the absence of RPE and accounted for a host of potential SPE confounds. This new surprise signal was evident in ventral striatum, primary sensory cortex, frontal poles, and amygdala. We interpret these findings via a normative model of surprise.
Laboratory for Social and Neural Systems Research, University of Zurich, Switzerland.
Justin R
JR
Chumbley
Christopher J
CJ
Burke
Klaas E
KE
Stephan
Karl J
KJ
Friston
Philippe N
PN
Tobler
Ernst
E
Fehr
D000328:Adult; D059031:Anticipation, Psychological / Q000502:physiology; D001921:Brain / Q000502:physiology*; D001931:Brain Mapping; D030141:Feedback, Psychological / Q000502:physiology*; D006801:Humans; D008279:Magnetic Resonance Imaging; D008297:Male; D008959:Models, Neurological; D009483:Neuropsychological Tests; D012054:Reinforcement, Psychology; D057566:Self Report; D055815:Young Adult
learning; prediction error; reward