Image-based nutritional assessment: Evaluating the performance of ChatGPT-4o on simple and complex meals

This study presents an exploratory pilot evaluation of the ChatGPT-4o artificial intelligence model in estimating energy and macronutrient content from images of meals with different levels of complexity: simple, moderate, and complex. Classifications were based on ingredient distinguishability, the presence of visually undetectable elements such as oil, sauce, or filling, and the overall variety of ingredients. Nine food items representing the three complexity levels were photographed under standardized natural lighting and uploaded to ChatGPT-4o. After initial predictions, additional content information was provided for selected samples, and second estimations were obtained. Predicted values were compared with reference values calculated by the researchers. Initial predictions showed substantial deviations, with errors reaching 54.4% for energy and 76.5% for fat, particularly in complex meals and those with visually obscured fat. Providing additional information was associated with improved accuracy for all macronutrients, with the R² for energy increasing from 0.591 to 0.941. Although these findings demonstrate that supplementary contextual information can enhance model performance, they represent preliminary, descriptive observations based on a small pilot dataset. Accordingly, the results should not be generalized or interpreted as evidence of readiness for real-world dietary planning, but rather as an initial feasibility indication requiring further validation with larger and more diverse samples. • ChatGPT-4o estimated energy and macronutrients from meal images of varying complexity. • Substantial prediction errors were observed, especially for fat content in complex meals. • Prediction accuracy significantly improved after providing additional contextual input. • Image-based AI systems may support nutritional assessment when combined with user input.