A cognitive neuroscience perspective on embodied language for human-robot cooperation.
This article addresses issues in embodied sentence processing from a "cognitive neural systems" approach that combines analysis of the behavior in question, analysis of the known neurophysiological bases of this behavior, and the synthesis of a neuro-computational model of embodied sentence processing that can be applied to and tested in the context of human-robot cooperative interaction. We propose a Hybrid Comprehension Model that links compact propositional representations of sentences and discourse with their temporal unfolding in situated simulations, under the control of grammar. The starting point is a model of grammatical construction processing which specifies the neural mechanisms by which language is a structured inventory of mappings from sentence to meaning. This model is then "embodied" in a perceptual-motor system (robot) which allows it access to sentence-perceptual representation pairs, and interaction with the world providing the basis for language acquisition. We then introduce a "simulation" capability, such that the robot has an internal representation of its interaction with the world. The control of this simulator and the associated representations present a number of interesting "neuro-technical" issues. First, the "simulator" has been liberated from real-time. It can run without being connected to current sensory motor experience. Second, "simulations" appear to be represented at different levels of detail. Our paper provides a framework for beginning to address the questions: how does language and its grammar control these aspects of simulation, what are the neurophysiological bases, and how can this be demonstrated in an artificial yet embodied cognitive system.
Madden
Carol
C
Stem Cell and Brain Research Institute, INSERM U846, 69676 Bron Cedex, France. carol.madden@inserm.fr
Hoen
Michel
M
Dominey
Peter Ford
PF
D032882:Comprehension; D003198:Computer Simulation; D006801:Humans; D007802:Language; D008037:Linguistics; D008960:Models, Psychological; D012371:Robotics / Q000379:methods*; D013997:Time Factors; D014584:User-Computer Interface; D014741:Video Recording