LG-FAL: Locality-customized GSA Federated Active Learning

Federated Learning (FL) is a learning paradigm which constructs machine learning models using decentralized datasets, with the goal of training a globally optimized model while preserving data privacy. In practice, annotating training data for FL is expensive and time-consuming. To solve this problem, researchers proposed to integrate active Learning (AL) strategy as Federated Active Learning (FAL) framework. However, existing research on FAL has two main limitations: (1) In the active learning part, more attention is paid to the global model, while ignoring the local information; (2) In the federated learning part, the popular Federated averaging (FedAvg) method relies on the assumption that the corresponding nodes in distributed neural networks share the same importance when averaging. In this paper, in order to tackle the two limitations, we propose a locality-customized GSA federated active learning (LG-FAL) method. Specifically, (i) both local and global information are taken into consideration to evaluate and select informative data samples to annotate for active learning; (ii) we integrate Gravitational Search Algorithm (GSA) to dynamically average local network parameters into the global parameter, by mimicking the principles of gravity and motion in the universe, for effective federated learning. As a result, LG-FAL can select a small subset of informative samples considering both local and global information, and at the same time, provide an improved trade-off between communication cost and learning accuracy. Experimental results show that LG-FAL significantly outperforms the current state-of-the-art baselines in terms of performance and effectiveness.