FedSR: A Semi-Decentralized Federated Learning Framework for Non-IID Data Based on Incremental Subgradient Optimization

In the Industrial Internet of Things (IoT), data heterogeneity across different devices poses a huge challenge to federated learning techniques, significantly reducing the performance of federated learning models. Additionally, the large number of devices participating in IoT federated learning and training imposes a substantial computational burden on cloud servers. Current federated learning research primarily adopts centralized or discentralized learning architectures, which cannot fundamentally solve these issues. To address this, we propose a novel semi-centralized cloud-edge-device hierarchical federate learning framework that integrated both centralized and decentralized federated learning approaches. Specifically, only a subset of adjacent devices forms small-scale ring clusters, and the cloud server aggregates the ring models to construct a global model. To mitigate the impact of data heterogeneity across devices, we use an incremental subgradient optimization algorithm within each ring cluster to enhance the generalization ability of the ring cluster models. Extensive experiments demonstrate that our approach effectively reduces the impact of data heterogeneity, improves model performance, and significantly alleviates the communication burden on cloud servers compared to centralized and discentralized federated learning frameworks. Indeed, the framework proposed in this paper aims to balance the strengths of centralized federated learning and ring federated learning. It achieves superior performance in addressing the data non-IID problem compared to centralized federated learning architectures while also mitigating issues associated with excessively large rings in ring architectures.