Model Level Contrastive Federated Learning with Differential Privacy

Federated Learning, as a typical paradigm of collaborative learning, effectively mediates the contradictions between model training and data privacy. However, the data heterogeneity and privacy leakage problems undermine the availability of federated learning. Existing studies focus solely on one of the problems. In this work, we manage to handle these two challenges simultaneously, by utilizing contrastive learning and differential privacy in local training, the federated learning system can stay safe and robust. The contrastive loss item pushes the local model towards the global model and a random noise generated by differential privacy is added to the model to protect the sensitive information hidden in the models. Extensive experiment results demonstrate that our method can exceed the baseline around 1%~2% in term of test accuracy.