title: Neural signature of hierarchically structured expectations predicts clustering and transfer of rule sets in reinforcement learning.
abstract: Often the world is structured such that distinct sensory contexts signify the same abstract rule set. Learning from feedback thus informs us not only about the value of stimulus-action associations but also about which rule set applies. Hierarchical clustering models suggest that learners discover structure in the environment, clustering distinct sensory events into a single latent rule set. Such structure enables a learner to transfer any newly acquired information to other contexts linked to the same rule set, and facilitates re-use of learned knowledge in novel contexts. Here, we show that humans exhibit this transfer, generalization and clustering during learning. Trial-by-trial model-based analysis of EEG signals revealed that subjects' reward expectations incorporated this hierarchical structure; these structured neural signals were predictive of behavioral transfer and clustering. These results further our understanding of how humans learn and generalize flexibly by building abstract, behaviorally relevant representations of the complex, high-dimensional sensory environment.
authors.affiliation: Department of Cognitive, Linguistic and Psychological Sciences, Brown Institute for Brain Science, Brown University, Providence, RI, USA; Department of Psychology, University of California Berkeley, Berkeley, CA, USA. Electronic address: annecollins@berkeley.edu.
authors.forename: Anne Gabrielle Eva
authors.initials: AGE
authors.lastname: Collins
authors.affiliation: Department of Cognitive, Linguistic and Psychological Sciences, Brown Institute for Brain Science, Brown University, Providence, RI, USA.
authors.forename: Michael Joshua
authors.initials: MJ
authors.lastname: Frank
mesh_terms: D000293:Adolescent; D000328:Adult; D001245:Association Learning / Q000502:physiology*; D002540:Cerebral Cortex / Q000502:physiology*; D016000:Cluster Analysis; D004569:Electroencephalography; D005260:Female; D006801:Humans; D008297:Male; D008960:Models, Psychological; D012054:Reinforcement, Psychology; D012201:Reward; D055815:Young Adult
keywords: Clustering; EEG; Prefrontal cortex; Structure-learning; Transfer