title: Dynamics of scene representations in the human brain revealed by magnetoencephalography and deep neural networks.
abstract: Human scene recognition is a rapid multistep process evolving over time from single scene image to spatial layout processing. We used multivariate pattern analyses on magnetoencephalography (MEG) data to unravel the time course of this cortical process. Following an early signal for lower-level visual analysis of single scenes at ~100ms, we found a marker of real-world scene size, i.e. spatial layout processing, at ~250ms indexing neural representations robust to changes in unrelated scene properties and viewing conditions. For a quantitative model of how scene size representations may arise in the brain, we compared MEG data to a deep neural network model trained on scene classification. Representations of scene size emerged intrinsically in the model, and resolved emerging neural scene size representation. Together our data provide a first description of an electrophysiological signal for layout processing in humans, and suggest that deep neural networks are a promising framework to investigate how spatial layout representations emerge in the human brain.
authors.affiliation: Department of Education and Psychology, Free University Berlin, Berlin, Germany; Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, MA, USA. Electronic address: rmcichy@mit.edu.
authors.forename: Radoslaw
authors.initials: R
authors.lastname: Martin Cichy
authors.affiliation: Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, MA, USA.
authors.forename: Aditya
authors.initials: A
authors.lastname: Khosla
authors.affiliation: McGovern Institute for Brain Research, MIT, Cambridge, MA, USA.
authors.forename: Dimitrios
authors.initials: D
authors.lastname: Pantazis
authors.affiliation: Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, MA, USA.
authors.forename: Aude
authors.initials: A
authors.lastname: Oliva
mesh_terms: D000328:Adult; D001931:Brain Mapping / Q000379:methods*; D002540:Cerebral Cortex / Q000502:physiology*; D005260:Female; D006801:Humans; D015225:Magnetoencephalography; D008297:Male; D008959:Models, Neurological; D015999:Multivariate Analysis; D016571:Neural Networks, Computer; D010364:Pattern Recognition, Visual / Q000502:physiology*; D010775:Photic Stimulation; D055815:Young Adult
keywords: Deep neural network; Magnetoencephalography; Representational similarity analysis; Scene perception; Spatial layout