OASG-Net: Occlusion Aware and Structure-Guided Network for Face De-Occlusion

During the COVID-19 coronavirus epidemic, almost everyone wears a facial mask, which poses a huge challenge for face recognition. Therefore, it is urgent to improve the performance of face de-occlusion of masked faces. However, previous inpainting approaches are limited as they require the knowledge of a given mask, and in the past there was also a lack of real-world masked face datasets suitable for the face de-occlusion task. To tackle above issues, we pioneer a real-world masked face de-occlusion dataset (RMFDD) with accurate mask labels. Further, we propose an occlusion aware and structure-guided network (OASG-Net) for face de-occlusion, consisting of mask prediction subnet, structure prediction subnet, and face de-occlusion subnet. In particular, due to the mask prediction subnet, OASG-Net achieves face de-occlusion without a given external mask. We also use face structure to guide OASG-Net, which makes the recovered face topology more natural and realistic. Besides, we design the mask aware layer to avoid the hard 0-1 mask updating of partial convolution in the face de-occlusion subnet. Extensive results on both face de-occlusion and face recognition tasks demonstrate the superiority of our OASG-Net over the state-of-the-art competitors. Code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/WHUfreeway/OASG-Net-Occlusion-Aware-and-Structure-Guided-Network-for-Face-De-Occlusion</uri>.