title: Gaussian mixture models and semantic gating improve reconstructions from human brain activity.
abstract: Better acquisition protocols and analysis techniques are making it possible to use fMRI to obtain highly detailed visualizations of brain processes. In particular we focus on the reconstruction of natural images from BOLD responses in visual cortex. We expand our linear Gaussian framework for percept decoding with Gaussian mixture models to better represent the prior distribution of natural images. Reconstruction of such images then boils down to probabilistic inference in a hybrid Bayesian network. In our set-up, different mixture components correspond to different character categories. Our framework can automatically infer higher-order semantic categories from lower-level brain areas. Furthermore, the framework can gate semantic information from higher-order brain areas to enforce the correct category during reconstruction. When categorical information is not available, we show that automatically learned clusters in the data give a similar improvement in reconstruction. The hybrid Bayesian network leads to highly accurate reconstructions in both supervised and unsupervised settings.
authors: Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen Nijmegen, Netherlands.
authors: Sanne
authors: S
authors: Schoenmakers
authors: Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen Nijmegen, Netherlands.
authors: Umut
authors: U
authors: Güçlü
authors: Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen Nijmegen, Netherlands.
authors: Marcel
authors: M
authors: van Gerven
authors: Institute for Computing and Information Sciences, Radboud University Nijmegen Nijmegen, Netherlands.
authors: Tom
authors: T
authors: Heskes
keywords: Bayesian network; data fusion; fMRI; probabilistic inference; reconstruction; semantic categories; unsupervised learning