Large-Scale Model Meets Federated Learning: A Hierarchical Hybrid Distributed Training Mechanism for Intelligent Intersection Large-Scale Model

The large-scale model (LSM) can handle large-scale data and complex problems, effectively improving the intelligence level of urban intersections. However, the traffic conditions at intersections are becoming increasingly complex, so the intelligent intersection LSMs (I <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> LSMs) also need to be continuously learned and updated. The traditional cloud-based training method incurs a significant amount of computational and storage overhead, and there is a risk of data leakage. The combination of edge artificial intelligence and federated learning provides an efficient and highly privacy protected computing mode. Therefore, we propose a hierarchical hybrid distributed training mechanism for I <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> LSM. Firstly, relying on the intelligent intersection system for cloud-network-terminal integration, we constructed an I <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> LSM hierarchical hybrid distributed training architecture. Then, we propose a hierarchical hybrid federated learning (H2Fed) algorithm that combines the advantages of centralized federated learning and decentralized federated learning. Further, we propose an adaptive compressed sensing algorithm to reduce the communication overhead. Finally, we analyze the convergence of the H2Fed algorithm. Experimental results show that the H2Fed algorithm reduces the communication overhead by 21.6% while ensuring the accuracy of the model.