A Study of Efficient Optimizer Configuration Strategies in Federated Learning for FedAvg and FedProx

Federated Learning (FL) is an effective way to perform decentralized model training while protecting data privacy. However, the choice of optimization algorithm can significantly impact its performance. This study focuses on the effectiveness of different combinations of FL frameworks (including Federated Averaging (FedAvg) and Federated Proximal (FedProx)) and optimizers (including Stochastic Gradient Descent (SGD) and Adaptive Momentum (Adam)) to determine the best strategies for improving convergence and accuracy. Specifically, this paper compares the performance of four algorithmic combination strategies, including FedAvg+SGD, FedAvg+Adam, FedProx+SGD, and FedProx+Adam, on the Federated Extended Modified National Institute of Standards and Technology (FEMNIST) dataset. The experiments evaluate key metrics such as test accuracy, communication efficiency, and convergence speed under non-independent and identically distributed (IID) data distributions. The experimental results show that the combination of FedProx and Adam achieves the highest test accuracy and faster convergence speed than other configurations. And FedAvg+SGD is still the configuration with the highest communication efficiency. The results show that adaptive optimizers such as Adam can improve FL performance when paired with a robust aggregation framework. The study's findings can provide practical insights for algorithm selection in joint learning scenarios.