Identifying thematic roles from neural representations measured by functional magnetic resonance imaging.
The generativity and complexity of human thought stem in large part from the ability to represent relations among concepts and form propositions. The current study reveals how a given object such as rabbit is neurally encoded differently and identifiably depending on whether it is an agent ("the rabbit punches the monkey") or a patient ("the monkey punches the rabbit"). Machine-learning classifiers were trained on functional magnetic resonance imaging (fMRI) data evoked by a set of short videos that conveyed agent-verb-patient propositions. When tested on a held-out video, the classifiers were able to reliably identify the thematic role of an object from its associated fMRI activation pattern. Moreover, when trained on one subset of the study participants, classifiers reliably identified the thematic roles in the data of a left-out participant (mean accuracy = .66), indicating that the neural representations of thematic roles were common across individuals.
a Center for Cognitive Brain Imaging, Department of Psychology , Carnegie Mellon University , Pittsburgh , USA.
Jing
J
Wang
a Center for Cognitive Brain Imaging, Department of Psychology , Carnegie Mellon University , Pittsburgh , USA.
Vladimir L
VL
Cherkassky
a Center for Cognitive Brain Imaging, Department of Psychology , Carnegie Mellon University , Pittsburgh , USA.
Ying
Y
Yang
b Language Technologies Institute, School of Computer Science , Carnegie Mellon University , Pittsburgh , USA.
Kai-Min Kevin
KM
Chang
a Center for Cognitive Brain Imaging, Department of Psychology , Carnegie Mellon University , Pittsburgh , USA.
Robert
R
Vargas
a Center for Cognitive Brain Imaging, Department of Psychology , Carnegie Mellon University , Pittsburgh , USA.
Nicholas
N
Diana
a Center for Cognitive Brain Imaging, Department of Psychology , Carnegie Mellon University , Pittsburgh , USA.
Marcel Adam
MA
Just
D001931:Brain Mapping / Q000706:statistics & numerical data*; D003210:Concept Formation / Q000502:physiology*; D006801:Humans; D007802:Language; D000069550:Machine Learning / Q000706:statistics & numerical data*; D008279:Magnetic Resonance Imaging / Q000706:statistics & numerical data*; D013850:Thinking / Q000502:physiology*
Functional magnetic resonance imaging; multivariate pattern analysis; propositional representation; thematic roles