title: Recognizing Materials using Perceptually Inspired Features.
abstract: Our world consists not only of objects and scenes but also of materials of various kinds. Being able to recognize the materials that surround us (e.g., plastic, glass, concrete) is important for humans as well as for computer vision systems. Unfortunately, materials have received little attention in the visual recognition literature, and very few computer vision systems have been designed specifically to recognize materials. In this paper, we present a system for recognizing material categories from single images. We propose a set of low and mid-level image features that are based on studies of human material recognition, and we combine these features using an SVM classifier. Our system outperforms a state-of-the-art system [Varma and Zisserman, 2009] on a challenging database of real-world material categories [Sharan et al., 2009]. When the performance of our system is compared directly to that of human observers, humans outperform our system quite easily. However, when we account for the local nature of our image features and the surface properties they measure (e.g., color, texture, local shape), our system rivals human performance. We suggest that future progress in material recognition will come from: (1) a deeper understanding of the role of non-local surface properties (e.g., extended highlights, object identity); and (2) efforts to model such non-local surface properties in images.
authors.affiliation: Disney Research, Pittsburgh, 4720 Forbes Avenue, Lower Level, Suite 110, Pittsburgh, PA 15213, lavanya@disneyresearch.com.
authors.forename: Lavanya
authors.initials: L
authors.lastname: Sharan
authors.forename: Ce
authors.initials: C
authors.lastname: Liu
authors.forename: Ruth
authors.initials: R
authors.lastname: Rosenholtz
authors.forename: Edward H
authors.initials: EH
authors.lastname: Adelson
keywords: Mechanical Turk; material classification; material recognition; perception; texture classification
references.citation: Nature. 2007 May 10;447(7141):206-9
references.pmid: 17443193
references.citation: Psychol Rev. 1999 Oct;106(4):795-834
references.pmid: 10560329
references.citation: IEEE Trans Pattern Anal Mach Intell. 2009 Nov;31(11):2032-47
references.pmid: 19762929
references.citation: J Vis. 2003;3(5):347-68
references.pmid: 12875632
references.citation: Psychol Sci. 2008 Feb;19(2):196-204
references.pmid: 18271869
references.citation: J Vis. 2003;3(8):541-53
references.pmid: 14632606
references.citation: IEEE Trans Pattern Anal Mach Intell. 2011 Dec;33(12):2368-82
references.pmid: 21709305
references.citation: J Vis. 2004 Sep 15;4(9):779-97
references.pmid: 15493970
references.citation: Nature. 1999 Dec 23-30;402(6764):877-9
references.pmid: 10622251
references.citation: Psychol Sci. 2004 Jan;15(1):33-9
references.pmid: 14717829
references.citation: J Opt Soc Am A Opt Image Sci Vis. 1998 Dec;15(12):2951-65
references.pmid: 9857525
references.citation: IEEE Trans Pattern Anal Mach Intell. 1986 Jun;8(6):679-98
references.pmid: 21869365
references.citation: IEEE Trans Biomed Eng. 2004 Dec;51(12):2148-59
references.pmid: 15605862
references.citation: Vis Neurosci. 2008 May-Jun;25(3):371-85
references.pmid: 18598406
references.citation: Biol Cybern. 1987;55(6):367-75
references.pmid: 3567240
references.citation: J Vis. 2004 Sep 23;4(9):798-820
references.pmid: 15493971
references.citation: Perception. 2005;34(5):565-75
references.pmid: 15991693
references.citation: J Opt Soc Am A Opt Image Sci Vis. 2008 Apr;25(4):846-65
references.pmid: 18382484