DRL-Based Joint Aggregation Frequency and Edge Association for Energy-Efficient Hierarchical Federated Learning

Hierarchical Federated Learning (HFL) has been proposed to achieve large-scale model training and more efficient communication, surpassing conventional Federated Learning (FL). However, inappropriate aggregation frequency and edge association in HFL result in excessive energy consumption for users with poor channels or hinder its convergence performance due to stochastic gradient descent (SGD) and Non-Independent and Identical Distribution (NIID) data, which is particularly challenging for energy-limited users. Motivated by this, a joint aggregation frequency and edge association optimization problem is proposed to minimize the long-term energy consumption during HFL training process. The problem can be formulated by incorporating computation, communication model and convergence analysis together. Due to the coupling between control variables, we decompose it into two sub-problems and adopt an iterative algorithm to approximate their optimal solutions. Specifically, the aggregation frequency is optimized under a given edge association by convex optimization to trade-off the computation and communication energy consumption, considering the convergence characteristic and SGD noise. Then, Deep Reinforcement Learning (DRL) is adopted to optimize edge association based on data distribution, dynamic channels and the derived aggregation frequency. Simulation results demonstrate that our proposed strategy achieves the lowest energy consumption while attaining the required model accuracy, outperforming other benchmarks.