title: Visual dictionaries as intermediate features in the human brain.
abstract: The human visual system is assumed to transform low level visual features to object and scene representations via features of intermediate complexity. How the brain computationally represents intermediate features is still unclear. To further elucidate this, we compared the biologically plausible HMAX model and Bag of Words (BoW) model from computer vision. Both these computational models use visual dictionaries, candidate features of intermediate complexity, to represent visual scenes, and the models have been proven effective in automatic object and scene recognition. These models however differ in the computation of visual dictionaries and pooling techniques. We investigated where in the brain and to what extent human fMRI responses to short video can be accounted for by multiple hierarchical levels of the HMAX and BoW models. Brain activity of 20 subjects obtained while viewing a short video clip was analyzed voxel-wise using a distance-based variation partitioning method. Results revealed that both HMAX and BoW explain a significant amount of brain activity in early visual regions V1, V2, and V3. However, BoW exhibits more consistency across subjects in accounting for brain activity compared to HMAX. Furthermore, visual dictionary representations by HMAX and BoW explain significantly some brain activity in higher areas which are believed to process intermediate features. Overall our results indicate that, although both HMAX and BoW account for activity in the human visual system, the BoW seems to more faithfully represent neural responses in low and intermediate level visual areas of the brain.
authors: Intelligent Systems Lab Amsterdam, Institute of Informatics, University of Amsterdam Amsterdam, Netherlands.
authors: Kandan
authors: K
authors: Ramakrishnan
authors: Cognitive Neuroscience Group, Department of Psychology, University of Amsterdam Amsterdam, Netherlands.
authors: H Steven
authors: HS
authors: Scholte
authors: Cognitive Neuroscience Group, Department of Psychology, University of Amsterdam Amsterdam, Netherlands.
authors: Iris I A
authors: II
authors: Groen
authors: Intelligent Systems Lab Amsterdam, Institute of Informatics, University of Amsterdam Amsterdam, Netherlands.
authors: Arnold W M
authors: AW
authors: Smeulders
authors: Intelligent Systems Lab Amsterdam, Institute of Informatics, University of Amsterdam Amsterdam, Netherlands ; Cognitive Neuroscience Group, Department of Psychology, University of Amsterdam Amsterdam, Netherlands.
authors: Sennay
authors: S
authors: Ghebreab
keywords: HMAX; bag of words; fMRI; low and intermediate features; representation similarity analysis; visual perception
references: Nat Neurosci. 1999 Nov;2(11):1019-25
references: 10526343
references: Cogn Psychol. 2000 Sep;41(2):176-210
references: 10968925
references: Neuroimage. 2001 Dec;14(6):1370-86
references: 11707093
references: J Cogn Neurosci. 2003 May 15;15(4):600-9
references: 12803970
references: J Physiol. 1962 Jan;160:106-54
references: 14449617
references: Science. 2004 Mar 12;303(5664):1634-40
references: 15016991
references: Prog Brain Res. 2006;155:23-36
references: 17027377
references: Ecology. 2006 Oct;87(10):2614-25
references: 17089669
references: J Vis. 2008 Jan 11;8(1):4.1-9
references: 18318607
references: Front Syst Neurosci. 2008 Nov 24;2:4
references: 19104670
references: Curr Biol. 2011 Oct 11;21(19):1641-6
references: 21945275
references: Neuroimage. 2012 Aug 15;62(2):782-90
references: 21979382
references: IEEE Trans Pattern Anal Mach Intell. 2012 Sep;34(9):1704-16
references: 22156101
references: Neuron. 2012 Jun 21;74(6):1099-113
references: 22726839
references: J Vis. 2013 Nov 22;13(13):25
references: 24273227
references: Neuroimage. 2014 Apr 15;90:449-68
references: 24389422
references: Proc Natl Acad Sci U S A. 2014 Jun 10;111(23):8619-24
references: 24812127
references: Biol Cybern. 1980;36(4):193-202
references: 7370364
references: Annu Rev Neurosci. 1996;19:109-39
references: 8833438
references: Annu Rev Neurosci. 1996;19:577-621
references: 8833455
references: Curr Opin Neurobiol. 1997 Aug;7(4):523-9
references: 9287204
references: Hum Brain Mapp. 1998;6(5-6):373-7
references: 9788075