Lossy Compression for Sparse Aggregation

In this paper, we investigate the efficient transmisSion of sparse models in a distributed learning system. The system consists of multiple clients, each possessing a sparse local model, and a central server responsible for aggregating the clients' models. Our target is to characterize the tradeoff between communication cost and accuracy in transmissions from the clients to the server. We propose a compression scheme that concatenates a universal covering code and an optimal source code. The numerical results demonstrate an improvement in the communication cost over previous findings in [1]–[3] by comparing with a lower bound on the communication cost derived using a variant of a generalized Fano's inequality.