MTRCP: Multimodal Two-Level Fusion Architecture for Roadside Cooperative Perception

With the rapid development of self-driving technologies, autonomous driving still faces challenges because of the complexity of traffic environments, making accurate and stable environmental perception crucial. Roadside units (RSUs) can significantly enhance the perception range of autonomous vehicles, effectively addressing blind spots and improving traffic safety. This article proposes the Multimodal Two-Level Fusion Architecture for Roadside Cooperative Perception, a novel framework that leverages RSUs to enable comprehensive and over-the-horizon perception capabilities. We fuse point clouds and images at the first level to generate local detection results and then perform a second-level fusion to integrate these local results, producing full-area perception. Experimental validation with the DAIR-V2X-seq dataset and data collected at the China Telecom Smart Grid Test Park demonstrates the effectiveness and feasibility of the proposed cooperative perception architecture.