Privacy-Preserving Facial Expression Reconstruction and De-identification in Masked Images

This paper introduces a novel approach for privately reconstructing facial expressions from masked image. The method focuses on enhancing the expression construct of the visible part of the face, mainly around the eyes. This is especially pertinent in the context of the pandemic where mask-wearing is common, allowing the method to operate continuously without the need for mask removal. Our approach starts by capturing masked images via a face scanning device, which are then processed for reconstruction and de-identification, while prioritizing user privacy by obscuring identity information. The core of this method involves a Generative Adversarial Network (GAN) model that reconstructs the facial structure from the area surrounding the eyes, followed by an encryptor model that converts the image into a secure latent code. This encrypted format protects the data during transmission and requires specific decryption protocols for developer access, ensuring the preservation of both emotional and identity features under authorized use. To the best of our knowledge, this is the first study to focus on privacy-preserving facial expression reconstruction and de-identification in masked image.