M4Net for occlusion-robust 3D dense face alignment and cross-subject expression transfer with 3DMM and mask augmentation

Abstract Self-occlusion and external occlusion significantly affect the accuracy of monocular 3D dense face alignment (3DDFA). Although the 3D Morphable Model (3DMM) provides a simple and effective facial prior, existing 3DMM-based methods still suffer from limited robustness caused by the scarcity of facial occlusion data. In this work, we present M4Net ( M ulti-scale features, M ulti-head outputs, M ask Augmented, and based on 3D M orphable Model), a 3DDFA framework that integrates multi-scale feature extractors, multi-head prediction, and a mask augmentation strategy. The proposed framework improves robustness under occluded conditions without increasing training cost. Moreover, by learning decoupled semantic tokens, M4Net disentangles the pose, shape, and expression of the head region. This decoupled representation not only facilitates occlusion-robust alignment but also enables flexible 3D expression, pose, shape, and texture transfer across different subjects without any extra training. Quantitative evaluations show that our approach achieves comparable accuracy on AFLW2000-3D (68 pts) and AFLW (21 pts), and improves the MAEV by up to 0.88% on AFLW2000-3D for head pose estimation. Qualitative results on five different datasets demonstrate the generalization and robustness of M4Net. In particular, facial expression transfer is a major highlight of this work.