LGP: Layerwise Gradient Purify for Robust Federated Learning Against Poisoning Attacks

Federated learning (FL) has become a promising framework for collaborative model training on devices while preserving privacy. However, despite its significant potential, it faces notable cyber threats, such as poisoning attacks and codenamed Byzantine clients. These threats have the potential to significantly degrade the global model by jeopardizing the integrity of the collaborative model training process. Whilst previous research has addressed the detection and elimination of malicious gradients from Byzantine clients, it has also shown that model poisoning attacks can evade most statistical defence approaches relying on metrics such as median and distance. To address the challenge posed by poisoning attacks, we introduce a novel approach called Layerwise Gradient Purify (<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LGP</i>), which aims to remove any harmful gradients before the global aggregation process. It is comprised of two closely-related stages. The first stage focuses on pruning gradients at the layer level using the Median Absolute Deviation (MAD) pruning criterion. In the second stage, statistical features are extracted from the pruned gradients layer-by-layer and then clustered into honest and malicious categories. The proposed methodology treats each layer of the model across all clients as a probability distribution, employing hierarchical clustering to differentiate between malicious and honest clusters. Moreover, we introduce a new innocent criterion for selecting honest clusters, relying on reputation scores and gradient deviations from the global model. Extensive experiments were conducted employing diverse deep learning models, including CNN, RNN, and MLP, across a spectrum of datasets, including Cifar-10, AG-News, MNIST and ToN-IoT. The experiments evaluated the resilience of state-of-the-art approaches against recently introduced attacks. The numerical results demonstrate that the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LGP</i> approach is effective and superior.