title: "Snap-n-Eat": Food Recognition and Nutrition Estimation on a Smartphone.
abstract: We present snap-n-eat, a mobile food recognition system. The system can recognize food and estimate the calorific and nutrition content of foods automatically without any user intervention. To identify food items, the user simply snaps a photo of the food plate. The system detects the salient region, crops its image, and subtracts the background accordingly. Hierarchical segmentation is performed to segment the image into regions. We then extract features at different locations and scales and classify these regions into different kinds of foods using a linear support vector machine classifier. In addition, the system determines the portion size which is then used to estimate the calorific and nutrition content of the food present on the plate. Previous approaches have mostly worked with either images captured in a lab setting, or they require additional user input (eg, user crop bounding boxes). Our system achieves automatic food detection and recognition in real-life settings containing cluttered backgrounds. When multiple food items appear in an image, our system can identify them and estimate their portion size simultaneously. We implemented this system as both an Android smartphone application and as a web service. In our experiments, we have achieved above 85% accuracy when detecting 15 different kinds of foods.
authors.affiliation: SRI International, Princeton, NJ, USA.
authors.forename: Weiyu
authors.initials: W
authors.lastname: Zhang
authors.affiliation: SRI International, Princeton, NJ, USA.
authors.forename: Qian
authors.initials: Q
authors.lastname: Yu
authors.affiliation: SRI International, Princeton, NJ, USA.
authors.forename: Behjat
authors.initials: B
authors.lastname: Siddiquie
authors.affiliation: SRI International, Princeton, NJ, USA ajay.divakaran@sri.com.
authors.forename: Ajay
authors.initials: A
authors.lastname: Divakaran
authors.affiliation: SRI International, Princeton, NJ, USA.
authors.forename: Harpreet
authors.initials: H
authors.lastname: Sawhney
mesh_terms: D000465:Algorithms; D003920:Diabetes Mellitus / Q000178:diet therapy*; D003927:Diet, Diabetic; D002149:Energy Intake; D005502:Food; D006801:Humans; D063731:Mobile Applications; D015596:Nutrition Assessment; D010781:Photography; D057285:Precision Medicine; D015203:Reproducibility of Results; D000068997:Smartphone; D060388:Support Vector Machine
keywords: food recognition; mobile food recognition; nutrition estimation; visual food recognition