CDL-H: Cluster-Based Decentralized Learning With Heterogeneity-Aware Strategies for Industrial Internet of Things

In the Industrial Internet of Things (IIoT), leveraging edge clients collaboratively for decentralized learning is increasingly promising for emerging intelligent applications. However, the resources of edge clients are typically heterogeneous, such as computing power and training data, leading to a significant decrease in both learning efficiency and accuracy. Furthermore, resource constraints on the central server restrict scaling to more clients and data. To address these challenges, this article introduces a novel framework called clustered-based decentralized learning with heterogeneity-aware strategies (CDL-H). The framework incorporates two key strategies: 1) client clustering strategy and 2) differentiated iteration strategy. Different from the traditional approaches, CDL-H divides the clients into multiple clusters based on the similarity of their data distribution. Within each cluster, edge clients initiate multiple local iterations based on local computation and exchange models with other clients without the need for a central server. Additionally, the theoretical analysis on the convergence rate of the proposed framework is provided. To validate its effectiveness, extensive experiments are implemented based on popular tasks, such as image classification and fault identification in time-series data. The results demonstrate the robustness of the CDL-H framework across a wide range of computing power heterogeneity, with differences as large as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$19 \times $ </tex-math></inline-formula> and dropout rate as high as 50%. Moreover, CDL-H outperforms existing baselines, with up to 9.93% accuracy improvement on Fashion-MNIST and up to 6.10% accuracy improvement on CWRU.