title: Brain-based decoding of mentally imagined film clips and sounds reveals experience-based information patterns in film professionals.
abstract: In the perceptual domain, it has been shown that the human brain is strongly shaped through experience, leading to expertise in highly-skilled professionals. What has remained unclear is whether specialization also shapes brain networks underlying mental imagery. In our fMRI study, we aimed to uncover modality-specific mental imagery specialization of film experts. Using multi-voxel pattern analysis we decoded from brain activity of professional cinematographers and sound designers whether they were imagining sounds or images of particular film clips. In each expert group distinct multi-voxel patterns, specific for the modality of their expertise, were found during classification of imagery modality. These patterns were mainly localized in the occipito-temporal and parietal cortex for cinematographers and in the auditory cortex for sound designers. We also found generalized patterns across perception and imagery that were distinct for the two expert groups: they involved frontal cortex for the cinematographers and temporal cortex for the sound designers. Notably, the mental representations of film clips and sounds of cinematographers contained information that went beyond modality-specificity. We were able to successfully decode the implicit presence of film genre from brain activity during mental imagery in cinematographers. The results extend existing neuroimaging literature on expertise into the domain of mental imagery and show that experience in visual versus auditory imagery can alter the representation of information in modality-specific association cortices.
authors.affiliation: NeuroCine, Department of Film, Television and Scenography, School of ARTS, Aalto University, FI-00076, Helsinki, Finland; Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, School of Science, Aalto University, FI-00076, Espoo, Finland; AMI Centre, AALTO NEUROIMAGING, Aalto University, FI-00076, Espoo, Finland. Electronic address: aline.deborst@maastrichtuniversity.nl.
authors.forename: Aline W
authors.initials: AW
authors.lastname: de Borst
authors.affiliation: Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, 6200 MD, Maastricht, the Netherlands.
authors.forename: Giancarlo
authors.initials: G
authors.lastname: Valente
authors.affiliation: Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, School of Science, Aalto University, FI-00076, Espoo, Finland.
authors.forename: Iiro P
authors.initials: IP
authors.lastname: Jääskeläinen
authors.affiliation: NeuroCine, Department of Film, Television and Scenography, School of ARTS, Aalto University, FI-00076, Helsinki, Finland; NeuroCine, Department of Media, School of ARTS, Aalto University, FI-00076, Helsinki, Finland.
authors.forename: Pia
authors.initials: P
authors.lastname: Tikka
mesh_terms: D000328:Adult; D001921:Brain / Q000502:physiology*; D001931:Brain Mapping; D005260:Female; D006801:Humans; D007092:Imagination / Q000502:physiology*; D008279:Magnetic Resonance Imaging; D008297:Male
keywords: Expertise; Film; MVPA; Mental imagery; Naturalistic stimuli; fMRI