Graph-Relational Federated Learning: Enhanced Personalization and Robustness

Hypernetwork has recently emerged as a promising technique to generate personalized models in federated learning (FL). However, existing works tend to treat each client equally and independently — each client contributes equally to learning the hypernetwork, and their representations are independent in the hypernetwork. Such an independent treatment ignores topological structures among different clients, which are usually reflected in the heterogeneity of client data distribution. In this work, we propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">panacea</i>, a novel FL framework that can incorporate client relations as a graph to facilitate learning and personalization by using graph hypernetwork. Empirically, we show <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">panacea</i> achieves state-of-the-art performance in terms of both accuracy and speed on multiple benchmarks. Further, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">panacea</i> improves the robustness by leveraging the client relation graph. Specifically, it (1) generalizes better to the novel clients outside of the training and (2) is more resilient to various adversarial attacks, including model poisoning and backdoor attacks, which is also proved by our theoretical analysis.