A Model-Prediction-Based Hierarchical Personalized Federated Learning Framework With Distributed Resource Optimization

With the number of users growing rapidly, the federated learning (FL) algorithm gradually moves towards a multi-layer framework to ensure learning performance. In this article, considering the non-independent and identically distributed scenario, a new hierarchical federated meta-learning (HFML) framework is studied. The Hessian-free Model-Agnostic Meta-Learning is introduced into our model to personalize the local models of edge users (EUs), which is more computationally efficient than the traditional meta-learning. To alleviate the learning performance reduction due to the scarce available bandwidth resources, a multilayer perceptron model prediction scheme based on the attention mechanism is deployed at the side of edge nodes (ENs). To achieve the tradeoff between learning time and model accuracy, the semi-synchronous cloud aggregation mechanism based on the learning states and parameter freshness is proposed. The convergence analysis of the proposed HFML algorithm is also provided to prove that the upper bound of the loss decay exists. To solve the complex nonconvex optimization problem whose target is to maximize the learning efficiency of HFML, considering device selection and communication resource allocation, a decentralized algorithm based on Jacobi-Proximal ADMM (JP-ADMM) is proposed. Extensive simulations are performed to demonstrate the effectiveness of the proposed method. Particularly, compared with the traditional hierarchical federated learning algorithm, the proposed HFML achieves better learning performance while reducing the latency.