Adaptive weighted aggregation federated learning algorithm based on dual regularization

Abstract In federated learning (FL), different clients have different goals and datasets. The server collaboratively trains a powerful global model by aggregating the model parameters trained locally by the client. Therefore, a crucial step is to aggregate multiple local models into a global model. Most baseline FL algorithms associate the aggregation weights with the size of the client’s data volume to generate a global model. However, due to the heterogeneity of edge computing systems and data, many methods have the problem of ‘client drift’, resulting in the final aggregation model not being the globally optimal model. This seriously hinders the convergence speed of the model’s global training. To address this issue, this paper proposes an adaptive aggregated federated learning method based on double regularization (FedCL), which takes the cosine similarity of the local model gradient as the initial weight for model aggregation and sets a trainable shrinkage factor to dynamically adjust the aggregation weight. Meanwhile, a constraint term has been added to the server-side proxy loss function. Train the coefficients of the constraint terms through the proxy dataset to adaptively control the intensity of the constraints. This causes the weight update amplitude of the model to tend to decrease, achieving the effect of double regularization. This paper conducted a large number of experiments on multiple datasets with different distributions and evaluated FedCL using various neural networks. The experimental results show that the method proposed in this paper can enhance the generalization ability of the global model on different datasets and neural networks, and outperforms the baseline federated learning method in non-IID setting.