Towards Federated Learning Over the Air: Why Scaling Up Helps?

Federated learning enables multiple clients to collaboratively train a common model while concurrently preserving data privacy. However, its performance is often constrained by limited communication resources, especially when the system encounters a large number of clients. Under those circumstances, integrating over-the-air computations into the model training procedure is considered an effective approach to coping with the communication bottleneck. Specifically, by uploading each client's intermediate parameters via analog transmissions instead of digital ones, the system can dramatically extend the number of clients it simultaneously supports in each communication round. However, that is achieved at the expense of introducing channel distortions, particularly fading and noise, in the aggregated global parameter. To demystify these effects, the present paper develops a theoretical framework to analyze the performance of the over-the-air federated model training process. Our analysis unveils a three-fold benefit from system scaling up, i.e., as the number of participating clients increases: (i) the privacy leakage, quantified by the mutual information between each client's locally possessed gradient and the edge's globally aggregated one, substantially decreases, (ii) the impairment of small-scale fading disappears due to the channel hardening effect, and (iii) the convergence rate is enhanced as thermal noise and gradient estimation error can be reduced. To that end, it establishes over-the-air model training as a viable approach for implementing federated learning in scenarios with a large number of clients. We corroborate the theoretical findings with extensive experiments.