title: Similarity-Based Fusion of MEG and fMRI Reveals Spatio-Temporal Dynamics in Human Cortex During Visual Object Recognition.
abstract: Every human cognitive function, such as visual object recognition, is realized in a complex spatio-temporal activity pattern in the brain. Current brain imaging techniques in isolation cannot resolve the brain's spatio-temporal dynamics, because they provide either high spatial or temporal resolution but not both. To overcome this limitation, we developed an integration approach that uses representational similarities to combine measurements of magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) to yield a spatially and temporally integrated characterization of neuronal activation. Applying this approach to 2 independent MEG-fMRI data sets, we observed that neural activity first emerged in the occipital pole at 50-80 ms, before spreading rapidly and progressively in the anterior direction along the ventral and dorsal visual streams. Further region-of-interest analyses established that dorsal and ventral regions showed MEG-fMRI correspondence in representations later than early visual cortex. Together, these results provide a novel and comprehensive, spatio-temporally resolved view of the rapid neural dynamics during the first few hundred milliseconds of object vision. They further demonstrate the feasibility of spatially unbiased representational similarity-based fusion of MEG and fMRI, promising new insights into how the brain computes complex cognitive functions.
authors.affiliation: Computer Science and Artificial Intelligence Laboratory and.|Department of Education and Psychology, Free University Berlin, Berlin, Germany.
authors.forename: Radoslaw Martin
authors.initials: RM
authors.lastname: Cichy
authors.affiliation: McGovern Institute for Brain Research, MIT, Cambridge, MA, USA.
authors.forename: Dimitrios
authors.initials: D
authors.lastname: Pantazis
authors.affiliation: Computer Science and Artificial Intelligence Laboratory and.
authors.forename: Aude
authors.initials: A
authors.lastname: Oliva
mesh_terms: D000328:Adult; D001931:Brain Mapping / Q000379:methods; D002540:Cerebral Cortex / Q000000981:diagnostic imaging / Q000502:physiology*; D005240:Feasibility Studies; D005260:Female; D006801:Humans; D008279:Magnetic Resonance Imaging / Q000379:methods; D015225:Magnetoencephalography / Q000379:methods; D008297:Male; D064847:Multimodal Imaging / Q000379:methods; D009483:Neuropsychological Tests; D010364:Pattern Recognition, Visual / Q000502:physiology*; D021641:Recognition, Psychology / Q000502:physiology*; D012815:Signal Processing, Computer-Assisted; D014795:Visual Pathways / Q000000981:diagnostic imaging / Q000502:physiology
keywords: MEG; fMRI; multimodal integration; representational similarity analysis; visual object recognition