Modeling body state-dependent multisensory integration.
The brain often integrates multisensory sources of information in a way that is close to the optimal according to Bayesian principles. Since sensory modalities are grounded in different, body-relative frames of reference, multisensory integration requires accurate transformations of information. We have shown experimentally, for example, that a rotating tactile stimulus on the palm of the right hand can influence the judgment of ambiguously rotating visual displays. Most significantly, this influence depended on the palm orientation: when facing upwards, a clockwise rotation on the palm yielded a clockwise visual judgment bias; when facing downwards, the same clockwise rotation yielded a counterclockwise bias. Thus, tactile rotation cues biased visual rotation judgment in a head-centered reference frame. Recently, we have generated a modular, multimodal arm model that is able to mimic aspects of such experiments. The model co-represents the state of an arm in several modalities, including a proprioceptive, joint angle modality as well as head-centered orientation and location modalities. Each modality represents each limb or joint separately. Sensory information from the different modalities is exchanged via local forward and inverse kinematic mappings. Also, re-afferent sensory feedback is anticipated and integrated via Kalman filtering. Information across modalities is integrated probabilistically via Bayesian-based plausibility estimates, continuously maintaining a consistent global arm state estimation. This architecture is thus able to model the described effect of posture-dependent motion cue integration: tactile and proprioceptive sensory information may yield top-down biases on visual processing. Equally, such information may influence top-down visual attention, expecting particular arm-dependent motion patterns. Current research implements such effects on visual processing and attention.
Department of Computer Science, University of Tübingen, Tübingen, Germany. martin.butz@uni-tuebingen.de
Martin V
MV
Butz
Anna
A
Belardinelli
Stephan
S
Ehrenfeld
D001288:Attention / Q000502:physiology; D006801:Humans; D007600:Judgment / Q000502:physiology*; D008663:Metacarpus / Q000294:innervation; D008954:Models, Biological; D009039:Motion Perception / Q000502:physiology*; D009949:Orientation; D010775:Photic Stimulation; D011187:Posture; D011336:Probability; D011434:Proprioception; D012399:Rotation; D013997:Time Factors; D014110:Touch / Q000502:physiology*