Self-Supervised Face Deocclusion via 3-D Face Reconstruction With Outlier Segmentation

Face occlusion poses a challenge for many human–machine systems, such as facial expression perception, social signal analysis, and human identity verification. Accurate face deocclusion is essential for improving the performance of identity recognition, expression recognition, and the robustness of human–machine systems. As a result, this area has attracted significant attention from researchers in recent years. However, most existing methods rely heavily on synthetic occluded face datasets and predefined occlusion masks labels, which limits their applicability in real-world scenarios. To this end, we propose a novel self-supervised generative adversarial networks (GANs)-based framework for face deocclusion in this study, which integrates 3-D facial reconstruction with outlier segmentation guidance. To achieve reliable self-supervised occlusion guidance, we introduce an outlier segmentation module that utilizes statistical priors to generate accurate occlusion masks, facilitating the deocclusion process. Furthermore, we design a GAN-based dual-branch module, which is capable of simultaneously generating the occlusion mask and the deoccluded face. Extensive experiments on the widely used datasets demonstrate the superior performance of our approach on existing methods. Our method achieves 35.71 in peak signal to noise ratio (PSNR) and 0.891 in structural similarity index measure (SSIM) for occluded face restoration, outperforming state-of-the-art techniques.