title: Decoding and encoding of visual patterns using magnetoencephalographic data represented in manifolds.
abstract: Visual decoding and encoding are crucial aspects in investigating the representation of visual information in the human brain. This paper proposes a bidirectional model for decoding and encoding of visual stimulus based on manifold representation of the temporal and spatial information extracted from magnetoencephalographic data. In the proposed decoding process, principal component analysis is applied to extract temporal principal components (TPCs) from the visual cortical activity estimated by a beamforming method. The spatial distribution of each TPC is in a high-dimensional space and can be mapped to the corresponding spatiotemporal component (STC) on a low-dimensional manifold. Once the linear mapping between the STC and the wavelet coefficients of the stimulus image is determined, the decoding process can synthesize an image resembling the stimulus image. The encoding process is performed by reversing the mapping or transformation in the decoding model and can predict the spatiotemporal brain activity from a stimulus image. In our experiments using visual stimuli containing eleven combinations of checkerboard patches, the information of spatial layout in the stimulus image was revealed in the embedded manifold. The correlation between the reconstructed and original images was 0.71 and the correlation map between the predicted and original brain activity was highly correlated to the map between the original brain activity for different stimuli (r=0.89). These results suggest that the temporal component is important in visual processing and manifolds can well represent the information related to visual perception.
authors: Department of Computer Science, National Chiao Tung University, Hsinchu, Taiwan. Electronic address: kuopc@cs.nctu.edu.tw.
authors: Po-Chih
authors: PC
authors: Kuo
authors: Department of Computer Science, National Chiao Tung University, Hsinchu, Taiwan; Institute of Biomedical Engineering, National Chiao Tung University, Hsinchu, Taiwan. Electronic address: yschen@cs.nctu.edu.tw.
authors: Yong-Sheng
authors: YS
authors: Chen
authors: Institute of Brain Science, National Yang-Ming University, Taipei, Taiwan; Integrated Brain Research Unit, Department of Medical Research, Taipei Veterans General Hospital, Taipei, Taiwan. Electronic address: lfchen@ym.edu.tw.
authors: Li-Fen
authors: LF
authors: Chen
authors: Institute of Brain Science, National Yang-Ming University, Taipei, Taiwan; Integrated Brain Research Unit, Department of Medical Research, Taipei Veterans General Hospital, Taipei, Taiwan. Electronic address: jchsieh@ym.edu.tw.
authors: Jen-Chuen
authors: JC
authors: Hsieh
mesh_terms: D000328:Adult; D005260:Female; D006801:Humans; D007091:Image Processing, Computer-Assisted; D015225:Magnetoencephalography / Q000379:methods*; D008297:Male; D008959:Models, Neurological; D010364:Pattern Recognition, Visual / Q000502:physiology*; D010775:Photic Stimulation; D025341:Principal Component Analysis; D014793:Visual Cortex / Q000502:physiology*; D055815:Young Adult
keywords: Magnetoencephalography; Manifold; Visual decoding; Visual encoding