Semantic Segmentation of Food Through Deep Learning: A Case Study

Semantic segmentation of food images is a challenging task due to the intrinsic features inherent in food. High inter-class similarity and high intra-class differences give rise to segmentation algorithm challenges. However implementing a high-performance model for food classification is beneficial for various scenarios in the food health field. Due to the lack of easy access to large, pixel-wide annotated datasets, research on semantic food image segmentation has hitherto been difficult. In this paper we present a performance evaluation of different deep-learning-based models applied to semantic food segmentation. Two existing datasets are selected (FoodSeg103 and UECFOODPIXCOMPLETE) and used as the basis for evaluation of model performance. Specifically, we comprehensively evaluate four models: Fully Convolutional Networks (FCN), Mask Recurrent Convolutional Neural Networks (Mask RCNN), DeepLabV3, and transformer-based models, Segformer. Our results show that FCN and SegFormer achieve the best overall accuracy at 85.9 % and 84.1% when applied to the UECFOODPIXCOMPLETE dataset for food segmentation.