A Classification Feedback GAN with Variational Color Shift for Conditional Synthesis of Hyper-class Scene Text Images

Scene text image synthesis is of great significance in studying conditional image generation. Image synthesis methods based on Generative Adversarial Networks (GAN) are widely adopted to generate text images with the advantage of low complexity compared to diffusion models. However, conditional generation by GAN for more than 10,000 categories remains challenging, and no existing models have addressed this problem. This study proposes a novel architecture named the Classification Feedback GAN based on Variational Color Shift (CFGAN-VCS), which employs a classification discriminator with a variational color augmentation module. This method simultaneously optimizes the classification loss for both real and fake images and employs embedded one-hot latent to implement conditional control, which can perform conditional generation of scene text with four English characters with 456,976 classes. Experiments demonstrate that this novel architecture can effectively generate hyper-class scene text images and provide a new direction for hyper-class conditional GANs.