title: Saliency-aware food image segmentation for personal dietary assessment using a wearable computer.
abstract: Image-based dietary assessment has recently received much attention in the community of obesity research. In this assessment, foods in digital pictures are specified, and their portion sizes (volumes) are estimated. Although manual processing is currently the most utilized method, image processing holds much promise since it may eventually lead to automatic dietary assessment. In this paper we study the problem of segmenting food objects from images. This segmentation is difficult because of various food types, shapes and colors, different decorating patterns on food containers, and occlusions of food and non-food objects. We propose a novel method based on a saliency-aware active contour model (ACM) for automatic food segmentation from images acquired by a wearable camera. An integrated saliency estimation approach based on food location priors and visual attention features is designed to produce a salient map of possible food regions in the input image. Next, a geometric contour primitive is generated and fitted to the salient map by means of multi-resolution optimization with respect to a set of affine and elastic transformation parameters. The food regions are then extracted after contour fitting. Our experiments using 60 food images showed that the proposed method achieved significantly higher accuracy in food segmentation when compared to conventional segmentation methods.
authors.affiliation: Department of Radiation Oncology, Washington University in Saint Louis, Saint Louis, MO, USA ; Department of Neurological Surgery, University of Pittsburgh, Pittsburgh, PA, USA.
authors.forename: Hsin-Chen
authors.initials: HC
authors.lastname: Chen
authors.affiliation: Department of Neurological Surgery, University of Pittsburgh, Pittsburgh, PA, USA.
authors.forename: Wenyan
authors.initials: W
authors.lastname: Jia
authors.affiliation: School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China.
authors.forename: Xin
authors.initials: X
authors.lastname: Sun
authors.affiliation: School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China.
authors.forename: Zhaoxin
authors.initials: Z
authors.lastname: Li
authors.affiliation: Department of Neurological Surgery, University of Pittsburgh, Pittsburgh, PA, USA.
authors.forename: Yuecheng
authors.initials: Y
authors.lastname: Li
authors.affiliation: Department of Psychiatry and Pharmacology, University of Pittsburgh, Pittsburgh, PA, USA.
authors.forename: John D
authors.initials: JD
authors.lastname: Fernstrom
authors.affiliation: Health and Community Systems, University of Pittsburgh, Pittsburgh, PA, USA.
authors.forename: Lora E
authors.initials: LE
authors.lastname: Burke
authors.affiliation: Department of Pediatrics, Baylor College of Medicine, Houston, TX, USA.
authors.forename: Thomas
authors.initials: T
authors.lastname: Baranowski
authors.affiliation: Department of Neurological Surgery, University of Pittsburgh, Pittsburgh, PA, USA ; Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA ; Department of Electrical Engineering, University of Pittsburgh, Pittsburgh, PA, USA.
authors.forename: Mingui
authors.initials: M
authors.lastname: Sun
keywords: active contour model; food segmentation; multi-resolution; quantitative dietary assessment; saliency map