A Close Look at the Communication Efficiency and the Energy Footprints of Robust Federated Learning in Industrial IoT

Federated learning (FL) can be used to distribute machine learning (ML) tasks across edge and Internet of Things (IoT) devices with limited resources. FL provides an alternative and much more practical solution to classical artificial intelligence (AI), which requires moving large data volumes to energy-hungry data centers. On the other hand, sustainability of FL processes should be accurately quantified as limiting energy consumption might require sacrificing accuracy. This article proposes a framework for real-time monitoring of energy and green house gas (GHG) emissions (carbon footprints) of FL systems. The framework is developed for both classical FL policies relying on the parameter server and emerging fully decentralized ones. The proposed approach considers, for the first time, the impact of ML model quantization and sparsification on the energy/carbon budget while also discussing novel gradient tracking (GT) FL strategies that are robust to data heterogeneity but require higher communication bandwidth. General guidelines for energy-efficient designs are discussed based on several case studies on real datasets. This article quantifies the energy footprint of continual FL processes that implement periodic adaptation on new data as foreseen by emerging IoT industry verticals. Results show that centralized FL is advantageous when strict carbon budgets are imposed or energy-inefficient (<50 Kbit/Joule) communication protocols are adopted. GT mechanisms are to be preferred in heterogeneous data environments and decentralized setups. Finally, using FL for continual model fine-tuning provides large energy savings (>80%), provided the ML model compression is properly tuned.