A Hybrid Model for Predicting Calories in Food Items via Image Acquisition and Classification Layers

Accurate calorie estimation from food images is essential for dietary monitoring and health management. Therefore, this research proposes a hybrid model that combines transformers and CNNs for predicting calorie values while utilizing the strengths of the former to model spatial connections and of the latter in feature extraction. The system uses a combined dataset, including five thousand unique real-world images, Open Food Facts, and Food-101, to ensure robustness in different food items and environments. Image augmentation and depth-based volume estimation are preprocessing techniques that enhance the efficacy of the model. The design has included a correlation layer, that translates features into calorie values, a transformer layer which understands the context, and a convolutional neural network layer to extract features. A complete analysis proves the exceptional performance of the model with a 98.2% accuracy and 7.24 ms mean absolute error on the test dataset. Real-time predictions can be generated using an interface, built on Streamlit. This outcome shows that the algorithm is theoretically capable of producing real-world, scalable estimates of calories.