Image Generation Algorithm of Text Description Based on StyleGAN

The training of the text-to-image task relies on a large number of image-text pairs, and the problems of insufficient sample size of the dataset and inprominent image subject content will greatly affect the overall performance of the model. In addition, most of the existing methods are stacked multi-stage generation structure, and the quality of generated images is difficult to meet the actual application requirements. Aiming at the above problems, this paper enhances the dataset and designs a single-stage text generation image model based on StyleGAN. Based on the analysis of global semantic features and semantic diversity expressions, this paper carries out adaptive perturbation text data enhancement, and designs an image cropping method that fuses boundary information, and adopts random cropping and rotation to complete image data enhancement. In the generator, the conditional style construction network fuses global semantic information and hidden space vectors to obtain conditional style control vectors that control different visual features of the image. The generation network adopts a progressive generation structure, accepts the control of conditional style in the way of AdaIN, and adds random noise to each layer at the same time, which improves the authenticity of the image and enriches the details. At the same time, the semantic inner product is added to the discriminator, and the loss function of cyclic consistency loss and visual semantic matching loss are introduced to further improve the semantic consistency of graphics and texts. The IS, FID and R-precision values of this model on the CUB-200-2011 dataset reached 4.96, 14.11 and 75.20%, respectively.

Authors

Institutions

Publication Details

Published
2024-09-13
DOI
https://doi.org/10.1145/3700906.3700908
Primary Topic
Simulation and Modeling Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Image Generation Algorithm of Text Description Based on StyleGAN

Pan Hao, Hui Dai
Simulation and Modeling Applications
article

Image Generation Algorithm of Text Description Based on StyleGAN

Pan Hao, Hui Dai
article en

Abstract

The training of the text-to-image task relies on a large number of image-text pairs, and the problems of insufficient sample size of the dataset and inprominent image subject content will greatly affect the overall performance of the model. In addition, most of the existing methods are stacked multi-stage generation structure, and the quality of generated images is difficult to meet the actual application requirements. Aiming at the above problems, this paper enhances the dataset and designs a single-stage text generation image model based on StyleGAN. Based on the analysis of global semantic features and semantic diversity expressions, this paper carries out adaptive perturbation text data enhancement, and designs an image cropping method that fuses boundary information, and adopts random cropping and rotation to complete image data enhancement. In the generator, the conditional style construction network fuses global semantic information and hidden space vectors to obtain conditional style control vectors that control different visual features of the image. The generation network adopts a progressive generation structure, accepts the control of conditional style in the way of AdaIN, and adds random noise to each layer at the same time, which improves the authenticity of the image and enriches the details. At the same time, the semantic inner product is added to the discriminator, and the loss function of cyclic consistency loss and visual semantic matching loss are introduced to further improve the semantic consistency of graphics and texts. The IS, FID and R-precision values of this model on the CUB-200-2011 dataset reached 4.96, 14.11 and 75.20%, respectively.

Wuhan University of Technology (CN)
Openalex Percentile: Top 74%
Simulation and Modeling Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Image Generation Algorithm of Text Description Based on StyleGAN — Pan Hao, Hui Dai · (2024) | TGRS Research Map | TGRS