Food Image Understanding to Nutrients and Calories for Allergies and Patients Support Using Deep Learning Techniques

The calories estimation and nutrients perception from real food images could help allergic persons, patients and health experts determine food choices and dietary patterns to avoid health risks more effectively. Most of the studies have estimated the food volume according to a registered known-size reference object associated with a pre-processed food image. In addition, an intensive focus was considered for the European and Asian cuisines. In this paper, a framework for calories estimation and nutrients prediction from real food images is proposed based on deep learning techniques, providing allergic people and patients an adequate support to better understand the nutritional content of their dietary choices and to make timely informative dietary decisions. The proposed framework utilizes CNNs, instance segmentation and semantic segmentation techniques to assess calories and predict nutrients based on the food volume in an instant real food image without a fiducial marker. Several experiments have been conducted on a new dataset for oriental and Mediterranean cuisine meals. The experimental results achieve an average accuracy level of 98 % and 70 % for defining food images and detecting food items respectively using the InceptionV3 model. This bridges the gap between visual food data and meaningful information to unlock valuable insights from food images, leading to advancements in dietary tracking and allergic support.