Federated Learning Optimization: Model Update Difference Minimization and Functional Based Adaptive Learning Rate Adjustment

The FedAdamPID algorithm resolves the overshoot problem suffered by the FedAdam algorithm under the framework of PID control. The research on FedAdamPID explores the connection between PID controllers and the process of federated learning algorithms, particularly applying it to the update rules on the server side. This study draws on the previous research of the research group, first establishing a connection between the local update process of the client and the PID controller, and then viewing the entire FL process involving both the client and the server as a multi-PID coupled system. On this basis, the minimum update difference and the dynamic adjustment strategy of the local learning rate are introduced, and an optimized algorithm, FedAdamPID-op, is proposed. Experiments on multiple standard datasets have verified that this optimized algorithm slightly outperforms FedAdamPID in terms of accuracy and the number of convergence rounds.