Catch Me If You Can: A Modified GAN Architecture Leveraging SIFT and Feature-matching for Highly Stable Training

Generative Adversarial Networks (GANs) traditionally consist of two networks: a generator that creates new samples and a discriminator that evaluates the authenticity of these samples. Both networks are trained together competitively to generate samples indistinguishable from real data. This paper proposes a novel GAN architecture, Frank-GAN, which comprises two generators and one discriminator. The inspiration for this work comes from the movie "Catch Me If You Can." In the film, the main character, Frank, deceives the government by forging checks and certifications. After apprehending him, the government decides to utilize his expertise by appointing him to banking security to catch other forgers. Similarly, the second generator in Frank-GAN is specially trained using the feature-matching technique to match the activations of hidden discriminator-layer feature values between real and fake images, to enhance the GAN's effectiveness and stability. Unlike traditional GANs that rely on random noise as input, the input to the second generator consists of keypoints that are extracted from real images using a Scale-invariant feature (SIFT). Frank-GAN was evaluated on the CelebA and The Oxford 102 Flower datasets. The Fréchet inception distance metric (FID) improved from 48.5 to 7.04 on the CelebA dataset and from 49.84 to 6.04 on the Oxford 102 Flower dataset, while the inception score (IS) increased significantly from 2.70 to 18.68 on the CelebA dataset, when using Frank_GAN compared to traditional DCGANs. This demonstrates the good performance of Frank-GAN in generating high-quality and realistic images at 128×128 size.