SPSA-based Consensus Algorithm for Collaborative Learning with Heavy-Tailed Noises

Collaborative learning is a fast growing branch of machine learning. This approach allows to learn a global model keeping user data private and to easily scale a system. However, many algorithms for collaborative learning rely on aggregation of parameters learned by an each individual agent on a single server followed by a global model obtaining. With such approach the server becomes a single failure point. Another common assumption concerns noises in data. Most of the works consider only noises with finite variance. Altough it is known, that heavy-tailed noises arise in many real models.