A Web-Based Food Image Classification System using an Ensemble of Multi-Color Space Parallel CNNs

There is a strong correlation between health and dietary choices. Thus, we must pay attention to our food to maintain a healthy diet. Whether you are a health-conscious individual trying to maintain a balanced lifestyle or a traveler eager to experience novel and exotic foods, it is important to be mindful of the food you consume. However, as new dishes are created, identifying food becomes increasingly difficult. Currently, the primary approaches to food image classification rely on deep learning models or ensembles of deep learning models that utilize single color space inputs, usually RGB images. This paper introduces a web-based food image classification system, DishNet, that employs a user-selectable ensemble of two-input parallel CNN models using multi-color spaces. Moreover, this system allows users to choose the voting mechanism for aggregating predictions and select the models to include in the ensemble. Our experiments on ensembles of two-input parallel CNN models yielded promising results. In particular, the ensemble involving the input pairs of RGB+XYZ, RGB+LUV, and LAB+LUV color spaces and using the product of probabilities voting method was the most effective combination because it attained an 85.16% accuracy on the Food-101 dataset.