Inverse retinotopy: inferring the visual content of images from brain activation patterns.
Traditional inference in neuroimaging consists in describing brain activations elicited and modulated by different kinds of stimuli. Recently, however, paradigms have been studied in which the converse operation is performed, thus inferring behavioral or mental states associated with activation images. Here, we use the well-known retinotopy of the visual cortex to infer the visual content of real or imaginary scenes from the brain activation patterns that they elicit. We present two decoding algorithms: an explicit technique, based on the current knowledge of the retinotopic structure of the visual areas, and an implicit technique, based on supervised classifiers. Both algorithms predicted the stimulus identity with significant accuracy. Furthermore, we extend this principle to mental imagery data: in five data sets, our algorithms could reconstruct and predict with significant accuracy a pattern imagined by the subjects.
Thirion
Bertrand
B
INRIA Futurs, Service Hospitalier Frédéric Joliot, 4, Place du Général Leclerc 91401 Orsay Cedex, France. bertrand.thirion@inria.fr
Duchesnay
Edouard
E
Hubbard
Edward
E
Dubois
Jessica
J
Poline
Jean-Baptiste
JB
Lebihan
Denis
D
Dehaene
Stanislas
S
D006801:Humans; D007092:Imagination; D008279:Magnetic Resonance Imaging; D008962:Models, Theoretical; D012160:Retina / Q000502:physiology*; D014793:Visual Cortex / Q000502:physiology*; D014796:Visual Perception / Q000502:physiology*