Embracing Diversity: Enhancing Generalization in Heterogeneous Federated Learning

With the advent of the big data era, deep learning technologies have made significant progress. However, in practical applications, data privacy protection and data dispersion have emerged as major challenges. Federated learning, as an emerging distributed learning framework, allows multiple participants to collaboratively train a global model while preserving data privacy. However, federated learning faces two major challenges in practical applications: data heterogeneity and catastrophic forgetting. This paper proposes a novel heterogeneous federated learning method that utilizes a federated cross-correlation matrix to enable knowledge communication at the logits level. This effectively promotes knowledge sharing and integration across different domains, thereby enhancing the model's generalization ability. To address the issue of catastrophic forgetting during local updates, this paper employs a heterogeneous non-target distillation method. By extracting useful knowledge from previous models, it effectively captures the relationships between domains and alleviates the problem of catastrophic forgetting. Experimental results on two image classification tasks (Digits and Office-Home) demonstrate that the proposed method outperforms existing methods in both inter-domain and intra-domain performance. Additionally, ablation studies validate the effectiveness of each module.