Boosting Federated Learning with Diffusion Models for Non-IID and Imbalanced Data

Federated learning (FL) has emerged as an effective paradigm in machine learning, enabling participants to extract value from diverse data sources while preserving privacy. However, FL faces significant challenges, including communication inefficiency, computational overhead, and data heterogeneity. The latter, in particular, can substantially impair FL performance by increasing convergence time and degrading generalization capabilities. Recent FL literature indicates that poor global model performance in heterogeneous conditions can be attributed to divergence between the classification layers of clients. To address this challenge, we propose a novel approach leveraging recent advancements in foundation models, specifically diffusion models, to generate synthetic data that bridges the gap between heterogeneous FL and centralized learning. Specifically, our method involves fine-tuning a diffusion model on the client side, using a small subset of images from the local dataset, to generate synthetic images. These images along with labels are shared with the server, where the final classification layer of a converged federated model is retrained to counteract classifier divergence. Our experiments demonstrate significant improvements, including a +30% increase in performance accuracy and a 9-fold acceleration in convergence for non-IID FL scenarios. Furthermore, we evaluate our method on real-world datasets prone to image artifacts and data imbalance, showcasing the effectiveness of our approach in both industrial and medical applications.