Text-to-Image Synthesis Combining Cascade Structure and Joint Attention

Since the text-to-image synthesis with non-stacked network structure only models the global features of the image and the semantic features of the text, it is easy to cause problems such as semantic inconsistency, loss of detail features, and incomplete main content in the generated images. To solve the above problems, this paper proposes a text-to-image generation method (cascaded structure and joint attention generative adversarial network (CSGAN)) that combines cascaded structure and joint attention. After encoding the text, this method utilizes a conditional enhancement module to process it, which enhances the expressive ability of the text features. In order to make the local area of the image better fit the text features, this method designs a joint attention module to solve the problem that the image generation process cannot fully reflect the local details of the text content. By using affine transform mapping visual features and cascade structure to fuse the features of different modules, the integrity of the whole content of the image is effectively guaranteed and the semantic consistency of the text image is improved. Experimental results on the CUB dataset show that compared with the current mainstream non-stacked network model DF-GAN model, the inception score index of the CSGAN model is improved by about 4.01%, and the Fréchet inception distance index is reduced by about 13.36%. These data indicators and visualization results fully demonstrate the effectiveness of the CSGAN model.