FedCosA: Optimised Federated Learning Model Using Cosine Annealing

The Federated Learning paradigm allows multiple devices or nodes to contribute to model training without sharing raw data, ensuring that sensitive information remains on the client side while still benefiting from collective learning. However, FL suffers several problems, including slow convergence, overfitting, and communication cost, particularly in Non-Independent and Identically Distributed (Non-IID) data settings. To solve these concerns, we offer an improved Federated Averaging (FedAvg) method that includes the Adam optimizer, cosine annealing learning rate scheduler, and weight decay (L2 regularisation). The Adam optimizer generates adaptive learning rates, but the cosine annealing schedule refines the learning rate over time for greater stability. Weight decay reduces overfitting by penalizing big model parameters. Furthermore, picking a subset of customers in each cycle lowers communication costs. Experimental findings on the MNIST dataset show that our strategy outperforms existing FL algorithms like FedAvg, FedSGD, FedNova, and FedSAM in test accuracy and loss minimization while significantly lowering communication load and training time. These enhancements make the suggested approach more effective and scalable for FL across several settings.