Improving Accuracy of Object Quantity and Text-Image Alignment Based on the Stable Diffusion Pretrained Model

In recent years, advancements in training methods using large-scale text-image datasets and the development of diffusion models have significantly improved the quality and efficiency of image generation from text prompts. Despite these strides, text-to-image generation models still face challenges in producing realistic images that accurately reflect the specified details in the prompts. A common issue with diffusion-based models is their difficulty in generating the precise number of objects described in the text. For instance, a prompt like "six tomatoes in a basket" may often result in an image with an incorrect number of tomatoes. To address this problem, this paper proposes an algorithm designed to enhance the accuracy of object quantity in generated images. The algorithm employs an object counting network to assess the number of objects in a given image and calculates the error between this predicted count and the expected count. This error gradient is then used to guide the predictions of the diffusion model, improving its ability to generate the specified number of objects accurately.