ELFS: Entropy-based Loss Function Selection for Global Model Accuracy in Federated Learning

Federated Learning (FL) is a paradigm that enables collaborative training while keeping data localized, avoiding direct sharing with a central server. However, Non-IID (Non-Independent and Identically Distributed) data across clients presents a significant challenge for FL, compromising the global model performance. In this paper, we propose Entropy-based Loss Function Selection (ELFS), designed to enhance global model accuracy by selectively adapting the loss function based on the entropy of each client’s data distribution. ELFS leverages two core steps before local training to determine the appropriate loss function: 1) entropy calculation and 2) adaptive loss function selection. The entropy calculation step quantifies the data distribution of each client based on label frequencies. Subsequently, in the adaptive loss function selection step, each client selects an appropriate loss function to mitigate the impact of data imbalance. Our experimental results on Non-IID datasets, CIFAR-10 and CIFAR-100, demonstrate that ELFS improves global model accuracy by up to 16.13% compared to conventional FL methods, such as FedAvg, FedProx, and FedPer. By optimizing entropy thresholds, we further demonstrate the importance of fine-tuning hyperparameters to maximize accuracy. Moreover, ELFS offers flexibility for integrating additional loss functions, providing potential for further performance improvements in handling Non-IID data.