Convergence-Guaranteed Federated Learning through Gradient Trajectory Smoothing with Triple-Objective Decomposition

Federated Learning (FL) has been widely adopted as a distributed machine learning paradigm aiming to derive a global model without transferring local data to the server. In the context of heterogeneous environments typical of many FL deployments, our research has identified the performance oscillation problem in existing FL methods, resulting in slow convergence and severe performance drop. In this article, we first investigate the global optimizing objective in FL and demonstrate that, due to data heterogeneity and partial client participation, the global updates in a single training epoch may diverge from the intended objectives of conventional FL methods. To address this problem, we introduce a triple-objective decomposition mechanism to decompose the overarching global objective into three distinct local objectives aimed at aligning client gradients. Subsequently, we propose a gradient trajectory smoothing technique known as FedGTS, which refines local updates by estimating a pseudo-gradient leveraging historical global update trajectories. This approach is designed to mitigate performance oscillations and enhance the stability of the learning process. We theoretically demonstrate that our approach reduces variance of local updates and achieves a guaranteed convergence rate. We experimentally show that the proposed method outperforms the baselines with faster convergence and higher accuracy. Extensive experiments validate the effectiveness of the proposed approach across various heterogeneity settings. Our codes are publicly available at GitHub ( https://github.com/ZongHR/FedGTS ).