title: Scene analysis in the natural environment.
abstract: The problem of scene analysis has been studied in a number of different fields over the past decades. These studies have led to important insights into problems of scene analysis, but not all of these insights are widely appreciated, and there remain critical shortcomings in current approaches that hinder further progress. Here we take the view that scene analysis is a universal problem solved by all animals, and that we can gain new insight by studying the problems that animals face in complex natural environments. In particular, the jumping spider, songbird, echolocating bat, and electric fish, all exhibit behaviors that require robust solutions to scene analysis problems encountered in the natural environment. By examining the behaviors of these seemingly disparate animals, we emerge with a framework for studying scene analysis comprising four essential properties: (1) the ability to solve ill-posed problems, (2) the ability to integrate and store information across time and modality, (3) efficient recovery and representation of 3D scene structure, and (4) the use of optimal motor actions for acquiring information to progress toward behavioral goals.
authors.affiliation: Department of Electrical Engineering and Computer Science, Case Western Reserve University Cleveland, OH, USA.
authors.forename: Michael S
authors.initials: MS
authors.lastname: Lewicki
authors.affiliation: Helen Wills Neuroscience Institute, School of Optometry, Redwood Center for Theoretical Neuroscience, University of California at Berkeley Berkeley, CA, USA.
authors.forename: Bruno A
authors.initials: BA
authors.lastname: Olshausen
authors.affiliation: Department of Biology, University of Southern Denmark Odense, Denmark.
authors.forename: Annemarie
authors.initials: A
authors.lastname: Surlykke
authors.affiliation: Department of Psychology and Institute for Systems Research, University of Maryland College Park, MD, USA.
authors.forename: Cynthia F
authors.initials: CF
authors.lastname: Moss
keywords: active perception; auditory streaming; echolocation; electroreception; neuroethology; scene analysis; top-down processes; vision
references.citation: An Acad Bras Cienc. 2004 Jun;76(2):267-73
references.pmid: 15258638
references.citation: J Comp Psychol. 2009 Nov;123(4):357-67
references.pmid: 19929104
references.citation: Behav Processes. 2005 Jun 30;69(3):331-41
references.pmid: 15896531
references.citation: Curr Opin Neurobiol. 2003 Apr;13(2):150-8
references.pmid: 12744967
references.citation: J Physiol Paris. 2002 Sep-Dec;96(5-6):405-19
references.pmid: 14692489
references.citation: Nature. 1983 Jun 23-29;303(5919):696-8
references.pmid: 6855915
references.citation: Curr Biol. 2011 Jul 26;21(14):R528-9
references.pmid: 21783026
references.citation: Neuron. 2012 Oct 18;76(2):266-80
references.pmid: 23083731
references.citation: Proc Biol Sci. 2004 May 7;271 Suppl 4:S167-9
references.pmid: 15252973
references.citation: Nature. 2005 May 12;435(7039):201-5
references.pmid: 15889091
references.citation: Nature. 2004 Jun 10;429(6992):657-61
references.pmid: 15190352
references.citation: Nature. 2013 Jan 3;493(7430):93-6
references.pmid: 23172147
references.citation: Anim Behav. 1998 Aug;56(2):467-477
references.pmid: 9787038
references.citation: J Comp Psychol. 2000 Mar;114(1):61-72
references.pmid: 10739312
references.citation: J Comp Psychol. 2008 Aug;122(3):264-73
references.pmid: 18729654
references.citation: Trends Cogn Sci. 1999 Sep;3(9):329-336
references.pmid: 10461195
references.citation: J Comp Physiol A Neuroethol Sens Neural Behav Physiol. 2011 May;197(5):491-503
references.pmid: 21246202
references.citation: Proc Biol Sci. 2004 Jul 22;271(1547):1467-75
references.pmid: 15306318
references.citation: J Exp Biol. 2009 May;212(Pt 9):1392-404
references.pmid: 19376960
references.citation: J Comp Physiol A Neuroethol Sens Neural Behav Physiol. 2006 Jun;192(6):601-12
references.pmid: 16645886
references.citation: Science. 2012 Jan 27;335(6067):469-71
references.pmid: 22282813
references.citation: Proc Natl Acad Sci U S A. 2011 Sep 13;108(37):E718-24
references.pmid: 21844350
references.citation: J Acoust Soc Am. 2001 Oct;110(4):2207-26
references.pmid: 11681397
references.citation: Philos Trans R Soc Lond B Biol Sci. 1997 Aug 29;352(1358):1231-9
references.pmid: 9304689
references.citation: Neural Comput. 1995 Nov;7(6):1129-59
references.pmid: 7584893
references.citation: Psychol Rev. 2007 Apr;114(2):340-75
references.pmid: 17500630
references.citation: J Comp Psychol. 1997 Mar;111(1):3-13
references.pmid: 9090135
references.citation: Nat Neurosci. 2008 Oct;11(10):1223-30
references.pmid: 18776895
references.citation: Vision Res. 2001 Nov;41(24):3145-61
references.pmid: 11711140
references.citation: Proc Natl Acad Sci U S A. 2007 Jul 17;104(29):12223-7
references.pmid: 17615235
references.citation: J Comp Psychol. 2004 Sep;118(3):325-31
references.pmid: 15482060
references.citation: Annu Rev Psychol. 1999;50:243-71
references.pmid: 10074679
references.citation: J Physiol Paris. 2004 Jan-Jun;98(1-3):67-80
references.pmid: 15477023
references.citation: Br J Psychol. 1958 Aug;49(3):182-94
references.pmid: 13572790
references.citation: J Physiol Paris. 2002 Sep-Dec;96(5-6):431-44
references.pmid: 14692491
references.citation: Trends Cogn Sci. 2010 Mar;14(3):138-46
references.pmid: 20138795
references.citation: Nature. 2005 Mar 17;434(7031):387-91
references.pmid: 15772663
references.citation: J Comp Physiol A. 1992 Feb;170(2):243-51
references.pmid: 1583608
references.citation: J Acoust Soc Am. 2006 Mar;119(3):1562-73
references.pmid: 16583901
references.citation: Neuron. 2010 Jan 28;65(2):150-64
references.pmid: 20152123
references.citation: Psychon Bull Rev. 2011 Feb;18(1):192-8
references.pmid: 21327347
references.citation: Ann N Y Acad Sci. 2008 Mar;1124:77-97
references.pmid: 18400925
references.citation: Atten Percept Psychophys. 2009 Oct;71(7):1439-59
references.pmid: 19801605
references.citation: Trends Cogn Sci. 2006 Jul;10(7):301-8
references.pmid: 16784882
references.citation: J Neurophysiol. 2006 Jun;95(6):3571-84
references.pmid: 16709722
references.citation: J Acoust Soc Am. 2007 May;121(5 Pt1):2842-51
references.pmid: 17550183
references.citation: Am Psychol. 1998 Jan;53(1):37-58
references.pmid: 9442582
references.citation: Neurosci Biobehav Rev. 2012 Feb;36(2):799-824
references.pmid: 22206900
references.citation: Q J Exp Psychol A. 2005 Jul;58(5):931-60
references.pmid: 16194942
references.citation: J Comp Psychol. 2008 Aug;122(3):235-51
references.pmid: 18729652
references.citation: J Vis. 2002;2(4):324-53
references.pmid: 12678582
references.citation: Psychol Rev. 1987 Apr;94(2):115-147
references.pmid: 3575582
references.citation: Phys Rev Lett. 2011 Nov 18;107(21):214301
references.pmid: 22181884
references.citation: IEEE Trans Pattern Anal Mach Intell. 2013 Jan;35(1):232-44
references.pmid: 22392707
references.citation: Philos Trans R Soc Lond B Biol Sci. 2008 Mar 12;363(1493):1011-21
references.pmid: 17827106
references.citation: Perception. 2004;33(9):1101-27
references.pmid: 15560510
references.citation: J Exp Biol. 1969 Nov;51(2):471-93
references.pmid: 5351426
references.citation: Front Behav Neurosci. 2010 May 28;4:26
references.pmid: 20577635
references.citation: Annu Rev Psychol. 2004;55:271-304
references.pmid: 14744217
references.citation: J Neurophysiol. 2004 Oct;92(4):2051-70
references.pmid: 15381741
references.citation: J Comp Physiol A Neuroethol Sens Neural Behav Physiol. 2009 Jun;195(6):557-69
references.pmid: 19347342
references.citation: J Comp Physiol A Neuroethol Sens Neural Behav Physiol. 2008 May;194(5):457-67
references.pmid: 18299849
references.citation: Annu Rev Entomol. 1996;41:287-308
references.pmid: 15012331
references.citation: J Exp Biol. 2007 Jul;210(Pt 13):2244-52
references.pmid: 17575030
references.citation: Philos Trans R Soc Lond B Biol Sci. 2011 Feb 27;366(1564):468-75
references.pmid: 21242136
references.citation: Proc Biol Sci. 2013 Jan 16;280(1754):20122830
references.pmid: 23325775
references.citation: Science. 2011 Jul 29;333(6042):627-30
references.pmid: 21798949
references.citation: J Comp Psychol. 2000 Jun;114(2):174-82
references.pmid: 10890589
references.citation: J Comp Physiol A. 1999 Oct;185(4):341-52
references.pmid: 10555268
references.citation: J Neurophysiol. 2004 Aug;92(2):1088-104
references.pmid: 15044521
references.citation: Philos Trans R Soc Lond B Biol Sci. 2011 Feb 27;366(1564):596-610
references.pmid: 21242146
references.citation: J Comp Neurol. 2008 Nov 20;511(3):342-59
references.pmid: 18803238
references.citation: Curr Biol. 2011 Jun 7;21(11):984-9
references.pmid: 21620708
references.citation: J Exp Biol. 2008 Jan;211(Pt 1):106-13
references.pmid: 18083738
references.citation: J Exp Biol. 2009 Apr;212(Pt 7):1011-20
references.pmid: 19282498
references.citation: Annu Rev Neurosci. 1999;22:319-49
references.pmid: 10202542
references.citation: Trends Cogn Sci. 2008 Dec;12(12):466-73
references.pmid: 18951831
references.citation: Vision Res. 2001 Mar;41(6):711-24
references.pmid: 11248261
references.citation: Nature. 2009 Jan 1;457(7225):83-6
references.pmid: 19020501
references.citation: Annu Rev Neurosci. 2007;30:1-30
references.pmid: 17335403
references.citation: Proc Natl Acad Sci U S A. 2010 Apr 13;107(15):7048-53
references.pmid: 20351291
references.citation: Vision Res. 2001;41(25-26):3559-65
references.pmid: 11718795
references.citation: Curr Biol. 2009 Dec 1;19(22):R1024-7
references.pmid: 19948136
references.citation: Front Behav Neurosci. 2010 Aug 05;4:
references.pmid: 20740076
references.citation: J Comp Physiol A Neuroethol Sens Neural Behav Physiol. 2006 Dec;192(12):1279-85
references.pmid: 16924503
references.citation: Trends Cogn Sci. 2007 Dec;11(12):520-7
references.pmid: 18024143
references.citation: IEEE Trans Pattern Anal Mach Intell. 2004 May;26(5):530-49
references.pmid: 15460277
references.citation: Psychol Rev. 1994 Apr;101(2):329-35
references.pmid: 8022964
references.citation: Vision Res. 2009 Jun;49(10):1286-94
references.pmid: 19138697
references.citation: Trends Cogn Sci. 2008 Oct;12(10):388-96
references.pmid: 18760955
references.citation: J Acoust Soc Am. 2003 Aug;114(2):1120-31
references.pmid: 12942989
references.citation: J Exp Biol. 2004 Nov;207(Pt 23):4121-33
references.pmid: 15498958