Seeing Through the Threat: Adaptive Attention-Guided Backdoor Defense in Non-IID Federated Learning

Federated Learning (FL) is a decentralized machine learning technique that trains models locally using only model parameters, reducing data transmission concerns and improving privacy. FL encounters security threats, particularly the Backdoor Attack, which injects targeted patterns by malicious clients during training. This can lead to incorrect predictions when exposed to specific triggers. Besides, the presence of Independent and Identically Distributed (non-IID) data in practical applications poses a significant challenge for FL, as conventional models consider data samples from the same distribution. Distinguishing non-IID distributions from malicious attacks, including model poisoning attacks, is particularly difficult, as they can induce gradient anomalies and affect global model accuracy. The issue of non-IID data and the prevention of misclassification remains a significant challenge. This work proposes an Adaptive Attention-Guided Multi-Teacher Distillation technique for non-IID FL settings. In this work, a student model acquires knowledge by selecting only those teachers who make accurate predictions and by extracting their consensus attention. This process utilizes adaptive attention to avoid the integration of backdoor-related attention, thereby mitigating the risks associated with backdoor attacks. Empirical results show that when testing on datasets comprising exclusively of samples with triggers, our method reliably maintains the Attack Success Rate (ASR) under 10%. In most cases, the proposed work demonstrates an average Accuracy in Trigger data (AiT) that outperforms existing state-of-the-art methods by approximately 9%. Thus, the proposed solution is more suitable for mitigating backdoor attacks in non-IID FL.