Erasing backdoor of deep neural networks using neural perturbation-based attention distillation

In the realm of safety-critical domains, the security of Deep Neural Networks (DNNs) is constantly challenged by the insidious threat of backdoor attacks. These attacks, which manipulate model outputs through malicious triggers, impede the development of DNNs in mission-critical applications. Despite the progress made by existing defense mechanisms, it remains unclear how to remove backdoor-related neurons from DNNs effectively. To address this challenge, we propose a novel eraser-based framework called Neural Perturbation-based Attention Distillation (NPAD). NPAD utilizes the ideas of neural perturbation and neural attention distillation. Initially, the teacher network perturbs the backdoor neurons to reveal their presence, followed by the targeted pruning operations. Subsequently, attention distillation is performed on the student network under the guidance of the adapted teacher network, thus maintaining its resilience against backdoor attacks and enhancing defense performance. During the knowledge transfer process, we introduce a weighted attention alignment mechanism to accelerate convergence during training, thereby achieving the resultant student network with heightened robustness. The experimental results clearly demonstrate that NPAD consistently outperforms a variety of existing state-of-the-art methods in mitigating the effects of backdoor attacks, where NPAD outperforms Neural Attention Distillation (NAD) (the best of the four defense methods) by 9.75% in the average reduction on the attack success rate of 10 backdoor attacks. Furthermore, the results also show that NPAD effectively eliminates backdoor triggers by utilizing a mere 1% of clean training data while simultaneously preventing any significant decline in performance on clean data.