Transformer-based multimodal deep learning and its application in flavor-nutrition synchronous evaluation

Flavor chemistry and nutritional quality are the core attributes determining the commercial value and consumer acceptance of ready-to-eat meals. This study developed a Transformer-based multimodal deep learning framework that integrates systematic flavor chemistry profiles, physicochemical nutrient indices, and visual features to achieve intelligent synchronous assessment of flavor and nutritional quality of ready-to-eat meals solely from product images. The model achieved 91.5% classification accuracy in distinguishing 32 ready-to-eat meal variants, comprising pork-, chicken-, and beef-based meals, and demonstrated strong predictive performance for key nutrients, with determination coefficients ( R 2 ) of 0.86 for protein, 0.80 for fat, and 0.79 for carbohydrate after weighted-Huber optimization, indicating high consistency with physicochemical measurements. Systematic flavor chemistry characterization via headspace solid-phase microextraction/gas chromatography-mass spectrometry (HS-SPME/GC-MS) identified 77 volatile flavor compounds, among which 22 key odor-active compounds with an odor activity value (OAV) ≥ 1 were screened, with aldehydes and spice-derived terpenes identified as major contributors to flavor differentiation among products. Consumer evaluation ( n = 100) revealed that flavor attributes dominated purchase decisions, and the model accurately identified core flavor descriptors of products. The proposed model provides a rapid image-based approach for predicting flavor-related attributes and nutritional quality of ready-to-eat meals. In the future, the framework could serve as an AI-assisted quality-control tool in centralized kitchens or ready-to-eat meal enterprises, while also being developed into a consumer-oriented AI application for image-based estimation of macronutrients and calories and personalized meal recommendations based on flavor preferences and nutritional needs.