Benchmarking Segmentation Models for Food Image Recognition

Accurate segmentations of food images are essential in digital nutrition tracking with relevance to chronic disease management and public global health. As diet-tracking apps gain popularity, the validity of such applications depends on pixel-accurate segmentations—particularly considering real foods are amorphous, overlapping, and irregular in appearance. Traditional heuristic- based methods struggle with such variability, emphasizing the need for robust computer vision models. This work systematically compares three convolutional neural network architectures—Fully Convolutional Networks (FCN), DeepLabV3, and Pyramid Scene Parsing Network (PSPNet)—on the FoodSeg103 dataset. The models are compared using the same training settings, including backbone architecture (ResNet-50), data augmentation methods, and optimization methods. The evaluations confirm FCN as the best-performing architecture with a better average Intersection over Union (mIoU) of 0.5281 compared to DeepLabV3 with a mIoU of 0.4478 and PSPNet with a mIoU of 0.3366. Notably, FCN outperforms the existing state-of-the-art FoodSAM model (mIoU of 0.464), indicating the strengths of optimized standard architectures in domain-specific applications like food segmentation. This work sets a solid foundation as a benchmark for food image segmentation and provides insights towards future work in automated nutrition monitoring and nutrition assessment systems.