SignDefence: Byzantine-Robust Federated Learning with Sign Direction and Leaky ReLU

The advancement of big data has paved the way for the development of intelligent and smart applications; however, privacy concerns often hinder fully realizing their benefits. Federated Learning (FL) has emerged as a promising framework for enhancing privacy while training models collaboratively across decentralized data sources. However, it remains susceptible to poisoning attacks, severely undermining its effectiveness. Existing robust aggregation techniques often struggle with the sensitivity of data distributions, and cluster-based strategies often fail to cluster poisoned model updates correctly. The direction obtained from the signs of the gradient mostly solves these problems but remains vulnerable to dying ReLU problems and usually becomes sensitive to outliers. In this paper, we introduce SignDefence, a sign direction and LeakyReLU-based aggregation technique, which considers the direction of the gradients and overcomes the performance issues related to the dying ReLU problem. Moreover, the proposed SignDefence computes Jaccard Similarity over binary encoded model weights and remains robust across sparse data. The experimental results suggest that the proposed technique shows consistently better accuracy and F1 score than the state-of-the-art techniques, without attack and under different attack scenarios.