Adaptive Learning Rate Optimizer In Hierarchical Clustering Asynchronous Federated Learning

The adaptive gradient descent algorithm in federated learning (FL) determines the speed of model convergence and has been widely studied and applied in recent years. Although the adaptive optimizer can automatically adjust the hyperparameters and find a good direction for the convergence of the model quickly, its applicability in heterogeneous environments is still lacking. In this study, the application scenario of adaptive optimizer AdaPID is expanded to heterogeneous condition, and an adaptive asynchronous hierarchical federated system is designed by improving the traditional asynchronous FL. After many experiments on standard data sets, the effectiveness of our work in dealing with system heterogeneity is confirmed.