title: Simultaneous modeling of visual saliency and value computation improves predictions of economic choice.
abstract: Many decisions we make require visually identifying and evaluating numerous alternatives quickly. These usually vary in reward, or value, and in low-level visual properties, such as saliency. Both saliency and value influence the final decision. In particular, saliency affects fixation locations and durations, which are predictive of choices. However, it is unknown how saliency propagates to the final decision. Moreover, the relative influence of saliency and value is unclear. Here we address these questions with an integrated model that combines a perceptual decision process about where and when to look with an economic decision process about what to choose. The perceptual decision process is modeled as a drift-diffusion model (DDM) process for each alternative. Using psychophysical data from a multiple-alternative, forced-choice task, in which subjects have to pick one food item from a crowded display via eye movements, we test four models where each DDM process is driven by (i) saliency or (ii) value alone or (iii) an additive or (iv) a multiplicative combination of both. We find that models including both saliency and value weighted in a one-third to two-thirds ratio (saliency-to-value) significantly outperform models based on either quantity alone. These eye fixation patterns modulate an economic decision process, also described as a DDM process driven by value. Our combined model quantitatively explains fixation patterns and choices with similar or better accuracy than previous models, suggesting that visual saliency has a smaller, but significant, influence than value and that saliency affects choices indirectly through perceptual decisions that modulate economic decisions.
authors.affiliation: Division of Biology and Division of Engineering and Applied Sciences, California Institute of Technology, Pasadena, CA 91125.
authors.forename: R Blythe
authors.initials: RB
authors.lastname: Towal
authors.forename: Milica
authors.initials: M
authors.lastname: Mormann
authors.forename: Christof
authors.initials: C
authors.lastname: Koch
mesh_terms: D000328:Adult; D000704:Analysis of Variance; D002755:Choice Behavior / Q000502:physiology*; D004467:Economics; D005133:Eye Movements / Q000502:physiology; D005260:Female; D006801:Humans; D008297:Male; D008960:Models, Psychological; D014796:Visual Perception / Q000502:physiology*
keywords: attention; eye tracking; multiple targets; preference; search
references.citation: Nat Rev Neurosci. 2001 Mar;2(3):194-203
references.pmid: 11256080
references.citation: Psychol Rev. 2005 Apr;112(2):291-328
references.pmid: 15783288
references.citation: J Neurophysiol. 1972 Jul;35(4):575-86
references.pmid: 4624741
references.citation: Nat Neurosci. 2010 Oct;13(10):1292-8
references.pmid: 20835253
references.citation: J Neurophysiol. 1976 Jul;39(4):722-44
references.pmid: 823302
references.citation: J Neurosci. 2003 Jul 23;23(16):6480-9
references.pmid: 12878689
references.citation: J Vis. 2011 Mar 10;11(3):
references.pmid: 21393388
references.citation: Front Hum Neurosci. 2010 Nov 17;4:208
references.pmid: 21267421
references.citation: Exp Brain Res. 1998 Jul;121(1):65-75
references.pmid: 9698192
references.citation: Neuron. 2007 Feb 15;53(4):605-17
references.pmid: 17296560
references.citation: Behav Brain Sci. 2003 Aug;26(4):445-76; discussion 477-526
references.pmid: 15067951
references.citation: Nat Neurosci. 2000 Aug;3(8):827-30
references.pmid: 10903577
references.citation: J Neurosci. 2008 May 28;28(22):5623-30
references.pmid: 18509023
references.citation: Vision Res. 2005 Mar;45(5):643-59
references.pmid: 15621181
references.citation: J Neurosci. 2011 Jan 5;31(1):331-8
references.pmid: 21209219
references.citation: Curr Biol. 2009 Mar 24;19(6):R247-8
references.pmid: 19321139
references.citation: Psychol Rev. 1988 Jan;95(1):15-48
references.pmid: 3353475
references.citation: J Neurosci. 2007 Sep 12;27(37):9984-8
references.pmid: 17855612
references.citation: Eur J Neurosci. 2011 Jun;33(11):1991-2002
references.pmid: 21645095
references.citation: Nature. 1996 Nov 7;384(6604):74-7
references.pmid: 8900281
references.citation: Nature. 1993 Dec 2;366(6454):467-9
references.pmid: 8247155
references.citation: Annu Rev Neurosci. 2010;33:1-21
references.pmid: 20192813
references.citation: Neuron. 2002 Oct 10;36(2):265-84
references.pmid: 12383781
references.citation: Trends Cogn Sci. 2000 Apr;4(4):138-147
references.pmid: 10740278
references.citation: J Cogn Neurosci. 2004 Dec;16(10):1695-705
references.pmid: 15701222
references.citation: Nat Neurosci. 2003 Dec;6(12):1317-22
references.pmid: 14608360
references.citation: J Opt Soc Am. 1962 Feb;52:210-3
references.pmid: 14006501
references.citation: Psychon Bull Rev. 2002 Sep;9(3):438-81
references.pmid: 12412886
references.citation: Front Neurosci. 2012 May 24;6:70
references.pmid: 22654730
references.citation: J Neurophysiol. 1998 Feb;79(2):817-34
references.pmid: 9463444
references.citation: J Neurosci. 2005 Jan 5;25(1):233-8
references.pmid: 15634786
references.citation: Science. 2009 May 1;324(5927):646-8
references.pmid: 19407204
references.citation: Psychol Rev. 2004 Jul;111(3):757-69
references.pmid: 15250782
references.citation: Science. 1996 Oct 18;274(5286):427-30
references.pmid: 8832893
references.citation: J Neurosci. 2012 Mar 7;32(10):3433-46
references.pmid: 22399766
references.citation: J Neurophysiol. 1998 Jun;79(6):3272-8
references.pmid: 9636126
references.citation: Nat Rev Neurosci. 2008 Jul;9(7):545-56
references.pmid: 18545266
references.citation: Psychol Rev. 1993 Jul;100(3):432-59
references.pmid: 8356185
references.citation: Nat Neurosci. 1999 Jun;2(6):549-54
references.pmid: 10448220
references.citation: Proc Natl Acad Sci U S A. 2011 Aug 16;108(33):13852-7
references.pmid: 21808009
references.citation: Psychon Bull Rev. 2000 Sep;7(3):424-65
references.pmid: 11082851
references.citation: Psychol Rev. 2006 Oct;113(4):700-65
references.pmid: 17014301
references.citation: Neuron. 2009 Sep 24;63(6):733-45
references.pmid: 19778504
references.citation: J Vis. 2007 Nov 21;7(14):4.1-17
references.pmid: 18217799
references.citation: Neural Netw. 2006 Nov;19(9):1395-407
references.pmid: 17098563
references.citation: J Neurosci. 2010 Jan 13;30(2):583-90
references.pmid: 20071521
references.citation: J Neurophysiol. 2007 Jan;97(1):942-7
references.pmid: 17079346
references.citation: Trends Cogn Sci. 2006 Aug;10(8):382-90
references.pmid: 16843702
references.citation: Science. 2004 Jul 9;305(5681):258-62
references.pmid: 15247483
references.citation: Prog Brain Res. 2005;147:251-62
references.pmid: 15581711
references.citation: J Vis. 2009 Mar 13;9(3):5.1-24
references.pmid: 19757944
references.citation: Nat Neurosci. 2006 Aug;9(8):1071-6
references.pmid: 16819520
references.citation: Trends Cogn Sci. 2007 Mar;11(3):118-25
references.pmid: 17276130
references.citation: Curr Biol. 2005 Mar 29;15(6):543-8
references.pmid: 15797023
references.citation: J Neurosci. 2011 Sep 14;31(37):13214-23
references.pmid: 21917804
references.citation: J Vis. 2012 Apr 01;12(4):9
references.pmid: 22510977
references.citation: Nature. 2006 May 11;441(7090):223-6
references.pmid: 16633341
references.citation: Trends Neurosci. 2004 Mar;27(3):161-8
references.pmid: 15036882
references.citation: Proc Natl Acad Sci U S A. 2010 Mar 16;107(11):5232-7
references.pmid: 20194768
references.citation: Annu Rev Neurosci. 2007;30:535-74
references.pmid: 17600525
references.citation: Acta Psychol (Amst). 2010 Oct;135(2):77-99
references.pmid: 20507828
references.citation: Nature. 1998 Jan 29;391(6666):481-4
references.pmid: 9461214
references.citation: Front Psychol. 2012 Jun 13;3:193
references.pmid: 22707945
references.citation: Psychol Rev. 2001 Jul;108(3):550-92
references.pmid: 11488378
references.citation: Nat Neurosci. 2003 Aug;6(8):891-8
references.pmid: 12858179
references.citation: Psychol Rev. 2007 Apr;114(2):376-97
references.pmid: 17500631
references.citation: J Neurophysiol. 2007 Feb;97(2):1756-74
references.pmid: 17122324
references.citation: J Neurosci. 2011 Jun 22;31(25):9298-306
references.pmid: 21697379