title: A systematic comparison between visual cues for boundary detection.
abstract: The detection of object boundaries is a critical first step for many visual processing tasks. Multiple cues (we consider luminance, color, motion and binocular disparity) available in the early visual system may signal object boundaries but little is known about their relative diagnosticity and how to optimally combine them for boundary detection. This study thus aims at understanding how early visual processes inform boundary detection in natural scenes. We collected color binocular video sequences of natural scenes to construct a video database. Each scene was annotated with two full sets of ground-truth contours (one set limited to object boundaries and another set which included all edges). We implemented an integrated computational model of early vision that spans all considered cues, and then assessed their diagnosticity by training machine learning classifiers on individual channels. Color and luminance were found to be most diagnostic while stereo and motion were least. Combining all cues yielded a significant improvement in accuracy beyond that of any cue in isolation. Furthermore, the accuracy of individual cues was found to be a poor predictor of their unique contribution for the combination. This result suggested a complex interaction between cues, which we further quantified using regularization techniques. Our systematic assessment of the accuracy of early vision models for boundary detection together with the resulting annotated video dataset should provide a useful benchmark towards the development of higher-level models of visual processing.
authors.affiliation: Brown University, Providence, RI 02912, United States; Department of Cognitive, Linguistic and Psychological Sciences, United States. Electronic address: david_mely@brown.edu.
authors.forename: David A
authors.initials: DA
authors.lastname: Mély
authors.affiliation: Brown University, Providence, RI 02912, United States; Department of Cognitive, Linguistic and Psychological Sciences, United States. Electronic address: junkyung_kim@brown.edu.
authors.forename: Junkyung
authors.initials: J
authors.lastname: Kim
authors.affiliation: Brown University, Providence, RI 02912, United States; Department of Cognitive, Linguistic and Psychological Sciences, United States. Electronic address: mmcgill@caltech.edu.
authors.forename: Mason
authors.initials: M
authors.lastname: McGill
authors.affiliation: Brown University, Providence, RI 02912, United States; Department of Engineering, United States. Electronic address: yuliang_guo@brown.edu.
authors.forename: Yuliang
authors.initials: Y
authors.lastname: Guo
authors.affiliation: Brown University, Providence, RI 02912, United States; Department of Cognitive, Linguistic and Psychological Sciences, United States; Brown Institute for Brain Science, United States. Electronic address: thomas_serre@brown.edu.
authors.forename: Thomas
authors.initials: T
authors.lastname: Serre
mesh_terms: D003118:Color Perception / Q000502:physiology*; D015350:Contrast Sensitivity / Q000502:physiology*; D003463:Cues; D006801:Humans; D008029:Lighting; D008962:Models, Theoretical; D009039:Motion Perception / Q000502:physiology*; D010364:Pattern Recognition, Visual / Q000502:physiology*; D015357:Vision Disparity / Q000502:physiology*; D014793:Visual Cortex / Q000502:physiology*
keywords: Boundary; Contour; Early vision; Grouping; Natural scenes; Primary visual cortex; Segmentation