Local Drift Correction and Attention Aggregation for Self-Organized Federated Learning

Federated learning (FL) is an emerging distributed machine learning paradigm that addresses the data silo problem while maintaining privacy protection. However, a primary challenge in FL is the significant statistical heterogeneity among clients’ local data distributions, which results in local data drift and subsequent global model performance degradation. To address this fundamental issue, we propose a framework termed FedSOB (Local Drift Correction and Attention Aggregation for Self-Organized Behaviors Federated Learning) in this paper. The FedSOB framework is inspired by the natural self-organized behavior of fish, achieving consistency in client behavior by introducing local drift correction and attention aggregation. Each client incorporates a momentum term during gradient descent, which encapsulates both the current gradient information and the global gradient information from preceding communication rounds, thereby modifying the gradient update process for clients. Furthermore, we have proposed an improved attention aggregation algorithm that mitigates the discrepancies caused by asynchronous model updates among clients, and have named it the Behavioral Attention Aggregation. Experimental results validate the efficacy of FedSOB, showing significant improvements in model accuracy, particularly in image classification tasks under non-IID data scenarios.