Towards Communication-Efficient Federated Learning via Loss-aware Quantization

A critical scalability challenge of Federated Learning (FL) lies in substantial communication overhead due to the transmission of massive parameters of local model updates between distributed clients and the central server. To alleviate this issue, numerous quantization techniques have been developed to compress local models prior to transmission. However, these methods inevitably result in performance degradation, struggling to meet the demands of high-compression scenarios. Motivated by this challenge, this paper proposes a novel quantization scheme, FedLQ, for communication-efficient FL. Unlike traditional quantization methods, FedLQ incorporates the quantized values into the local loss function to capture the impacts of quantized values on the learning performance. Building upon this, we propose a loss-aware quantization strategy to optimize the quantized values by minimizing a joint objective of training and quantization losses, which efficiently reduces transmission cost while mitigating learning performance degradation. Moreover, an inner-outer loop optimization algorithm is designed to solve this problem. Extensive experiment results demonstrate that FedLQ achieves almost the same performance as vanilla FL, even under the quantization scenarios with small quantization levels.