FedSparse: A Communication-Efficient Federated Learning Framework Based on Sparse Updates

Federated learning (FL) strikes a balance between privacy preservation and collaborative model training. However, the periodic transmission of model updates or parameters from each client to the federated server incurs substantial communication overhead, especially for participants with limited network bandwidth. This overhead significantly hampers the practical applicability of FL in real-world scenarios. To address this challenge, we propose FedSparse, an innovative sparse communication framework designed to enhance communication efficiency. The core idea behind FedSparse is to introduce a communication overhead regularization term into the client’s objective function, thereby reducing the number of parameters that need to be transmitted. FedSparse incorporates a Resource Optimization Proximal (ROP) term and an Importance-based Regularization Weighting (IRW) mechanism into the client update objective function. The local update process optimizes both the empirical risk and communication overhead by applying a sparse regularization weighted by update importance. By making minimal modifications to traditional FL approaches, FedSparse effectively reduces the number of parameters transmitted, thereby decreasing the communication overhead. We evaluate the effectiveness of FedSparse through experiments on various datasets under non-independent and identically distributed (non-IID) conditions, demonstrating its flexibility in resource-constrained environments. On the MNIST, Fashion-MNIST, and CIFAR datasets, FedSparse reduces the communication overhead by 24%, 17%, and 5%, respectively, compared to the baseline algorithm, while maintaining similar model performance. Additionally, on simulated non-IID datasets, FedSparse achieves a 6% to 8% reduction in communication resource consumption. By adjusting the sparsity intensity hyperparameter, we demonstrate that FedSparse can be tailored to different FL applications with varying communication resource constraints. Finally, ablation studies highlight the individual contributions of the ROP and IRW modules to the overall improvement in communication efficiency.