Federated Class-Incremental Learning via Weighted Aggregation and Distillation

Federated Class-Incremental Learning (FCIL) aims to design privacy-preserving collaborative training methods to continuously learn new classes from distributed datasets. In these scenarios, federated clients face the challenge of encountering new classes while being constrained by limited memory capacity, which can lead to catastrophic forgetting in the resulting global model. Existing FCIL approaches tend to overlook the challenges posed by the heterogeneity of dataset label distribution among clients, thereby constraining the generalization capacity of the global model they learn. Some of these methods also suffer from excessive computational burdens when addressing catastrophic forgetting problems. Furthermore, certain approaches are constrained to handling only straightforward data, posing significant difficulties in managing complex datasets and tackling more intricate scenarios. In this article, we propose the Weighted Aggregation and Distillation-based FCIL (<monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">WAD-FCIL</monospace>) method to address these limitations. To address data heterogeneity arising from class imbalance, we first introduce a task-aware client clustering method to identify clients with extreme class deviations before global model aggregation to eliminate potential impact on the global model. Then, we propose a multisampling weighted aggregation approach during the global FL model update that integrates knowledge from different clients and dynamically adjusts the weight of each client model to facilitate model update. To mitigate catastrophic forgetting, we propose a multimodel distillation strategy that involves selecting multiple teacher models for knowledge distillation. Extensive experiments comparing <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">WAD-FCIL</monospace> with ten state-of-the-art methods demonstrate that it significantly outperforms the baselines by 0.8%–3.2% in terms of average test accuracy on three representative benchmark datasets. The code of this work is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/wufeng10010/WAD-FCIL</uri>.