Cuisine Image Synthesis with Improved Multiscale GANs Guided by CLIP

In this study, a novel approach using a multiscale generative adversarial network (GAN) was proposed to generate high-quality, realistic cuisine images from ingredient and instruction texts. A model, FoodGAN-v2, that integrated text and image features within a hierarchical structure was also proposed for generating images at multiple resolutions. The proposed approach leveraged the capabilities of multiscale GANs to capture intricate details and variations required for realistic food imagery. FoodGAN was enhanced by incorporating the contrastive language-image pretraining (CLIP) model to guide image generation. Using CLIP as both a sentence encoder and a contrastive loss function, the generated images were realistic and accurately aligned with their textual descriptions. This study offers an efficient and effective solution for generating cuisine images, thereby significantly contributing to computational food analysis. FoodGAN offers three major advantages: (1) it produces detailed and realistic images that accurately represent complex cuisine compositions; (2) it efficiently integrates text and image features, ensuring that generated images closely align with their corresponding textual descriptions; and (3) it promotes advancements in the field of computational food analysis by addressing unique challenges associated with generating realistic cuisine images by integrating CLIP for enhanced guidance and precision.