Keypoint-guided for adversarial patch defence in vision transformer-based face recognition

Adversarial patch attacks represent a novel security threat in the field of face recognition, with existing defense methods predominantly relying on a two-stage localization process, often at the expense of conventional face recognition performance. To enhance robustness against adversarial patch attacks in face recognition, we propose a keypoint-guided defense method. This method leverages facial prior knowledge and the robustness of the ViT network structure, localizing patches through keypoint guidance and enhancing occlusion robustness via a random dropout strategy, thereby effectively defending against adversarial patch attacks. Experiments demonstrate our method's excellence against various white-box adversarial patch attacks, achieving over 95% [email protected] and above 98% accuracy in most scenarios. It maintains the original model's high accuracy on clean datasets without needing adversarial samples for training. Extensive tests confirm its effectiveness in resisting attacks while preserving accuracy, highlighting its practical applicability.