Fruit weight estimation using image-based deep learning for use in dietary assessment

• Convolutional Neural Networks (CNNs) have substantially advanced image recognition • DL models integration into dietary assessment tools enhances fruit intake accuracy • Image datasets specifically created to include representative range of fruit weight • DL models were trained on diverse fruit images allowing real-world settings One of the key limitations of current dietary assessment methodologies is to adjust the food portion size. In this study, we propose a Deep Learning (DL) approach to estimate the weight of individual pieces of commonly consumed fruits (apple, pear, orange and banana) from single-view RGB (Red, Green and Blue) photographs. The DL models developed in this study were based on convolutional neural networks trained on an ad-hoc dataset of 48,960 photographs including a wide and representative range of fruit piece weights, reflecting typical market variability and reducing the likelihood of encountering out-of-range samples. The photographs of apples (n=12,960), pears (n=17,208), oranges (n=8,712) and bananas (n=10,080) were taken under different conditions. The DL models were evaluated in terms of saliency maps and regression metrics. Mean Absolute Error (MAE) values indicated that the measurements in the DL models developed would be out —as a mean— by no more than 20.57 g (apple), 19.25 g (pear), 28.31 g (orange) and 21.93 g (bananas) between the predicted and the observed values, quite acceptable considering the variability in fruit weights and photographic conditions. The four DL models developed in this study predict fruit weight from a simple photograph, requiring only a visible €1 coin as a reference, without considering the background. This approach is feasible because our DL models were trained on diverse images across different angles, distances, lighting conditions, tablecloths and dish types, allowing the models to generalise well in real-world contexts.