A Linearly Convergent Quantized Stochastic Gradient Tracking Method With Variance Reduction

In recent years, there has been a growing demand for decentralized algorithms that enable agents to solve optimization problems with lower communication and computational complexity. We propose Q-SGT, the quantized stochastic gradient tracking algorithm that incorporates variance reduction, specifically designed to address finite-sum optimization problems. To enhance communication efficiency, we develop a dynamic quantization method, where a scaling factor is generated adaptively to scale the data to a suitable range for quantization, thereby avoiding saturation and dead zone. Additionally, the bit allocation of the quantizer is dynamically adjusted to reduce quantization noise. To the best of our knowledge, this is the first study that achieves linear convergence to the optimal solution in the presence of both quantization noise and imprecise first-order gradient information under a constant stepsize for decentralized optimization problems. Numerical experiments further demonstrate that Q-SGT can substantially reduce communication overhead without compromising convergence performance.