Global representation fine-tuning for federated self-supervised representation learning

Federated self-supervised representation learning combines federated learning with self-supervised mechanisms to learn general representations from distributed unlabeled data, effectively reducing reliance on labeled data. However, under data heterogeneity, existing methods primarily focus on aligning local and global models in the parameter space, often overlooking the issue of knowledge forgetting in the global model caused by incompatibility in local representation spaces. This limits the quality of representations and overall model performance. To address this challenge, we propose FedGRF, a global representation fine-tuning method for federated self-supervised representation learning. FedGRF maintains a generator on the server to produce pseudo-data, which is then used to drive global representation fine-tuning and mitigate the forgetting of local representation knowledge by the global model. By mining hard samples arising from the fusion of local representations and employing a controllable fine-tuning mechanism, FedGRF effectively promotes the transfer of local representation knowledge to the global model. Extensive experimental results demonstrate that FedGRF achieves competitive performance improvements over existing methods. • FedGRF is proposed under the framework of federated self-supervised representation learning (FedSSRL), which fine-tunes the global representation in a data-driven manner on the server side to effectively alleviate the problem of local representation knowledge forgetting caused by direct model aggregation. • An algorithm that mines hard samples arising from the fusion of local representations is designed to facilitate effective transfer of local representation knowledge to the global model. • A controllable fine-tuning mechanism is introduced into FedGRF to enhance the stability and effectiveness of the fine-tuning process.