Multi-Modal Federated Learning Based Resources Convergence for Satellite-Ground Twin Networks

Satellite-ground twin networks (SGTNs) are regarded as a promising service paradigm, which can provide mega access services and powerful computation offloading capabilities via cloud-fog automation functions. Specifically, cloud-fog automation technologies are collaboratively leveraged to enable dense connectivity, pervasive computing, and intelligent control in terrestrial industrial cyber-physical systems, whose system-level privacy security can be strengthened via blockchain based consensus protocol. Moreover, digital twin (DT) can shorten the gap between physical unities and digital space to enable instant data mapping in SGTNs environments. However, complex multi-modal network environments, such as stochastic task size, dynamic low earth orbit location, and time-varying channel gains, hinder better performance metrics in terms of energy consumption, throughput and privacy overhead. Hence, we establish a SGTN integrated cloud-fog automation model to transfer task data to low earth orbit satellites, and then execute broad communication access, powerful computation offloading, and efficient twin control. Next, we propose a Lyapunov stability theory based multi-modal federated learning (LST-MMFL) method to optimize the battery energy, the size of block, computation frequency, and the number of twin control for minimizing the total energy consumption and privacy overhead. Furthermore, we design a novel blockchain based transaction verification protocol to strengthen privacy security, derive performance upper bounds of SGTN model, and fulfill the long-term average task as well as energy queue constraints. Finally, massive simulation results show that the proposed LST-MMFL algorithm outperforms existing state-of-the-art benchmarks in line with energy consumption, available battery level, networked control and privacy protection overhead.