Classification Performance Optimization Method Based on Semi-supervised Federated Learning and Dual-domain Knowledge Distillation

Federated learning has emerged as a significant distributed learning paradigm, allowing devices to locally train models and transmit the updated parameters to a central server for aggregation, enabling collaborative model updates. However, data heterogeneity and catastrophic forgetting bring distinctive challenges. First, in non-lID (identically and independently distributed) data environments, model performance in other domains deteriorates significantly. Second, during local updates, models are optimized solely on local data, which leads to overfitting to the current data distribution and forgetting previously learned knowledge, resulting in catastrophic forgetting. Finally, traditional federated learning methods require the transmission of large model parameters, which results in high communication overhead. To address these challenges, we propose a classification performance optimization method based on semi-supervised federated learning and dual-domain knowledge distillation. For the data heterogeneity problem, we leverage unlabeled public data to enhance the model's generalization capability across different data distributions, thereby improving classifier performance. To mitigate catastrophic forgetting, we incorporate knowledge distillation in local updates, facilitating both inter and intra domain information sharing while preserving privacy. Experimental results demonstrate that, compared to typical federated learning methods, our method achieves better classification performance with lower communication overhead.