Face Frontalization Method with 3D Technology and a Side-Face-Alignment Generative Adversarial Networks

When the pose of a face changes, the facial features may be misaligned or shifted, thus reducing the quality of facial details. Therefore, face images with different angles affect face recognition, expression synthesis, style migration, and other related studies. To address this problem, we propose a face frontalization conversion model based on 3D technology and generative adversarial networks. The model first performs face fitting via 3DFFA, which is rotated and rendered in 3D space to generate training data pairs. These paired training data are then fed into a generative adversarial network. We use a modified CirGAN generator and a multiscale discriminator for training, and non-frontal face images from natural environments are fed into the generator to generate frontal face images. The multiscale discriminator is used to judge the quality of the generated images. Next, the Facenet and MTCNN are used to extract the features of the converted face and obtain the recognition results of the different angles of the face to verify the model's conversion effect. We experimented with the model on Multiple and CFP datasets, and the results show that the non-frontal face recognition results are improved compared with VGG-FACE, HPN, TP-GAN, and CAPGGAN.