Resilient Federated Learning Using Trimmed-Clipping Aggregation

Distributed learning such as federated learning has received increased attention from the research community due to its ability to accelerate learning on large data. Being vulnerable to attacks, ensuring resilience of such learning has become a pivotal research area. Various model poisoning attacks exist in the literature, including state-of-the-art time-coupled attacks that have proven many of the existing resilient algorithms unsuccessful. In this work, we propose a novel aggregation technique that is resilient against such poisoning attacks. This method, called trimmed clipping, uses the concept of using history in the learning by storing the aggregated gradient at the parameter server from the last epoch. The aggregation technique combines a trimming algorithm, which is based on gradient similarity and uses the last aggregated gradient as the reference, and a clipping operation. Even though the aggregation method was developed particularly to defend against time-coupled attack, it also provides resilience against other simple poisoning attacks. Our analysis formally prove resilience of trimmed clipping against adversarial workers, and convergence of the federated learning algorithm using this aggregation technique. Empirical evaluation on three standard tasks demonstrate the effectiveness of our method against various attacks including time-coupled attacks and simple model poisoning attacks. Trimmed clipping aggregation shows improved resilience compared to other existing aggregations against time-coupled attacks in terms of tolerable adversarial workers, while providing satisfactory performance against other attacks as well.