title: Crossmodal Association of Visual and Haptic Material Properties of Objects in the Monkey Ventral Visual Cortex.
abstract: Just by looking at an object, we can recognize its non-visual properties, such as hardness. The visual recognition of non-visual object properties is generally accurate [1], and influences actions toward the object [2]. Recent studies suggest that, in the primate brain, this may involve the ventral visual cortex, which represents objects in a way that reflects not only visual but also non-visual object properties, such as haptic roughness, hardness, and weight [3-7]. This new insight raises a fundamental question: how does the visual cortex come to represent non-visual properties--knowledge that cannot be acquired directly through vision? Here we addressed this unresolved question using fMRI in macaque monkeys. Specifically, we explored whether and how simple visuo-haptic experience--just seeing and touching objects made of various materials--can shape representational content in the visual cortex. We measured brain activity evoked by viewing images of objects before and after the monkeys acquired the visuo-haptic experience and decoded the representational space from the activity patterns [8]. We show that simple long-term visuo-haptic experience greatly impacts representation in the posterior inferior temporal cortex, the higher ventral visual cortex. After the experience, but not before, the activity pattern in this region well reflected the haptic material properties of the experienced objects. Our results suggest that neural representation of non-visual object properties in the visual cortex emerges through long-term crossmodal exposure to objects. This highlights the importance of unsupervised learning of crossmodal associations through everyday experience [9-12] for shaping representation in the visual cortex.
authors.affiliation: Division of Sensory and Cognitive Information, National Institute for Physiological Sciences, Okazaki 444-8585, Japan; Department of Physiological Sciences, SOKENDAI (The Graduate University for Advanced Studies), Okazaki 444-8585, Japan. Electronic address: ngoda@nips.ac.jp.
authors.forename: Naokazu
authors.initials: N
authors.lastname: Goda
authors.affiliation: Division of Sensory and Cognitive Information, National Institute for Physiological Sciences, Okazaki 444-8585, Japan; Department of Physiological Sciences, SOKENDAI (The Graduate University for Advanced Studies), Okazaki 444-8585, Japan.
authors.forename: Isao
authors.initials: I
authors.lastname: Yokoi
authors.affiliation: Department of Histology and Neurobiology, Dokkyo Medical University, Tochigi 321-0293, Japan.
authors.forename: Atsumichi
authors.initials: A
authors.lastname: Tachibana
authors.affiliation: Department of Molecular Neuroimaging, National Institute of Radiological Sciences, Chiba 263-8555, Japan.
authors.forename: Takafumi
authors.initials: T
authors.lastname: Minamimoto
authors.affiliation: Division of Sensory and Cognitive Information, National Institute for Physiological Sciences, Okazaki 444-8585, Japan; Department of Physiological Sciences, SOKENDAI (The Graduate University for Advanced Studies), Okazaki 444-8585, Japan.
authors.forename: Hidehiko
authors.initials: H
authors.lastname: Komatsu
mesh_terms: D000818:Animals; D000067290:Behavior Rating Scale; D001931:Brain Mapping; D008251:Macaca / Q000502:physiology*; D008279:Magnetic Resonance Imaging; D009474:Neurons / Q000166:cytology; D046709:Pattern Recognition, Physiological; D010364:Pattern Recognition, Visual; D013702:Temporal Lobe / Q000166:cytology / Q000502:physiology*; D014110:Touch; D014793:Visual Cortex / Q000502:physiology*; D014796:Visual Perception