Federated Unlearning With Fast Recovery

Recent federated unlearning studies mainly focus on removing the target client's contributions from the global model permanently. However, the requirement for accommodating temporary user exits or additions in federated learning has been neglected. In this paper, we propose a novel recoverable federated unlearning scheme, named RFUL, which allows users to remove or add their local model to the global one at any time easily and quickly. It mainly consists of two main components, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.,</i> knowledge unlearning and knowledge recovery. In knowledge unlearning, the target contributions can be eliminated by training with mislabeled target data, while preserving the non-target contributions through distillation using the original model. In knowledge recovery, the forgotten contributions can be restored by training the target data using classification loss, while the non-target contributions are maintained through feature distillation and parameter freezing on the classifier. Both knowledge unlearning and recovery processes only require the participation of target data, guaranteeing the algorithm's practicality in federated learning systems. Extensive experiments demonstrate the significant efficacy of RFUL. For knowledge unlearning, RFUL matches state-of-the-art methods using only target data, achieving a runtime speedup of 3.3 to 8.7 times compared to retraining across various datasets. For knowledge recovery, RFUL exceeds state-of-the-art incremental learning methods by 5.02% to 29.97% in accuracy and achieves a runtime speedup of 1.8 to 4.4 times compared to retraining on different datasets.