Nutriconv: multitask learning framework for digital dietary tracking trained on EFSA’s pancake dataset

Abstract The growing prevalence of nutrition-related health conditions calls for advanced tools to support reliable and efficient dietary monitoring. This paper presents NutriConv, a lightweight multitask convolutional neural network designed to simultaneously perform food classification and weight estimation from single-item food images. Trained on the institutionally validated PANCAKE dataset from the European Food Safety Authority, NutriConv combines classification and regression objectives within a unified architecture, optimized via a hybrid loss function. While its classification accuracy remains lower than that of specialized single-task models, NutriConv achieves competitive regression performance and offers a practical balance between both tasks. Its compact design enables deployment on resource-constrained platforms such as smartglasses and mobile health devices, expanding its usability in real-world dietary tracking scenarios. Extensive experiments confirm its robustness, including external validation on the Nutrition5K dataset, underscoring the model’s generalizability. This work highlights the potential of multitask learning for integrated, scalable, and accessible AI-based nutrition assessment.