Teleportation Links: Mitigating Catastrophic Forgetting in Decentralized Federated Learning

Decentralized approaches are inspired by the self-organizing principles observed in natural and social systems. These methods offer a scalable and resilient framework for collaborative learning. Decentralized federated learning (DFL) uses these principles to avoid the dependency on a central controller. However, a key challenge arises from data heterogeneity. This often leads to catastrophic forgetting, where the model significantly loses its ability to remember and use previously learned information. This loss of knowledge can reduce the reliability of DFL in critical applications, such as autonomous systems, financial services, energy management, and transportation networks. To tackle these challenges, in this paper, we investigate how data distribution affects DFL performance, with a focus on how bias propagates across nodes. We also explore how local model learning rates affect the trade-off between learning stability and convergence speed. At the same time, we evaluate the performance drops at individual nodes. Furthermore, to improve connectivity and speed up knowledge sharing, we propose adding a limited number of teleportation links, which aim to reduce the average distance between pairs of nodes. Extensive experimental results demonstrate the effectiveness of this strategy, showing reduced catastrophic forgetting, faster convergence, and improved resilience across various scenarios.