QSFL: Two-Level Communication-Efficient Federated Learning on Mobile Edge Devices

In cross-device horizontal federated learning (FL), the communication cost of transmitting complete models between edge devices and a central server is a significant bottleneck, due to expensive, unreliable, and low-bandwidth wireless connections. As a solution, we propose a novel <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FL</u> framework named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">QSFL</i>, towards <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">optimizing FL uplink (client-to-server) communication at both client and model levels</i>. At the client level, we design a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Qualification Judgment (<u>Q</u>J)</i> algorithm to sample high-qualification clients to upload models. At the model level, we design a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Sparse Cyclic Sliding Segmentation (<u>S</u>CSS)</i> algorithm to further compress the local model transmitted from the client to the server in the uplink communication. We prove that QSFL can converge over wall-to-wall time, and develop an optimal hyperparameter searching algorithm based on theoretical analysis to enable QSFL to make the best trade-off between model accuracy and communication cost. Experimental results show that QSFL achieves state-of-the-art compression ratios with marginal model accuracy degradation. Since mobile edge devices as FL clients often have heterogeneous system resources, such as communication bandwidth, we propose two novel <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">dynamic segmentation strategies with varied counts or sizes</i> based on QSFL to enhance the robustness of QSFL to FL system heterogeneity. For some mobile edge devices joining as FL clients with both limited uplink and downlink communication bandwidths, they can not pull up the global model from the server. To tackle it, we propose a novel <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">symmetric downlink compression</i> scheme on top of QSFL to further reduce the downlink (server-to-client) communication costs, hence enabling a bidirectional communication-efficient FL. Theory analysis and experiments demonstrate that QSFL with dynamic segmentation or symmetric downlink compression still keeps convergence and takes a better trade-off between model accuracy and communication efficiency than without them.