Face frontalization GAN with self-supervised learning for pose-invariant facial recognition

Despite impressive milestones achieved by generative models such as Generative Adversarial Network (GAN) in synthesizing faces, maintaining face identity remains a major challenge for most face recognition system. In this paper, we propose a Self-supervised Face Frontalization GAN (SF-GAN), which produces a frontal face image from a profile image. We designed a profile method to preserve identity information and fine details in high-resolution that focus on pose feature space, which helps SF-GAN reconstruct high-frequency face texture details. SF-GAN switches key facial regions between the input image and its reconstructed version to produce more realistic images. Furthermore, we introduce an orthogonal constraint to our generator that penalizes redundant latent representations, resulting in a more diverse pose appearance details. While the discriminator is designed with a mechanism of facial awareness for each facial region to improve overall model performance. FaceNet and MTCNN are then used to extract correlated face features in order to determine the accuracy of face recognition. Our evaluations are based on the Multi-PIE and CAS-PEAL datasets. Results showed that SF-GAN achieved a better face quality visualization and rank-1 face identification.