ABD-HFL: Byzantine-resistant Decentralized Hierarchical Federated Learning

Hierarchical federated learning (HFL) has attracted academic attention to improve the efficiency of federated learning (FL) in real-world applications, however, little research has been done to explore the structural advantages of HFL against Byzantine attacks and to investigate how to make HFL immune to top-level server Single Point of Failure (SPOF). To explore this field and improve the robustness of HFL, we propose a novel generalized paradigm ABD-HFL for asynchronous Byzantine-resistant decentralized hierarchical federated learning, a multi-tier structure without a central server for FL tasks with a large number of devices. Based on the layered structure, an innovative universal Byzantine resistance mechanism is designed in ABD-HFL, which enables it to apply a combination of multiple Byzantine robust techniques, making ABD-HFL more powerful than any single application of such techniques. Besides, ABD-HFL is a fully decentralized HFL, there is no central server, but rather multiple nodes at the top level agree on the global model where malicious model updates are excluded. A new concept of pipeline learning workflow is also introduced to study communication efficiency in ABD-HFL, which is based on asynchronous communication between various levels to train and propagate the global model. Our numerical evaluation validates the advantage of ABD-HFL in terms of robustness and communication efficiency.