Heterogeneous Multiscale Cooperative Perception for Connected Autonomous Vehicles via V2X Interaction

Connected autonomous driving and vehicle-toeverything (V2X) technology brings new opportunities for precise perception in occluded and sight-limited environments. The emergence of cooperative perception via V2X interaction has a positive impact on the safe and efficient driving of connected autonomous vehicles (CAVs). However, faced with the diversity and heterogeneity of V2X data, how to effectively process it to achieve better cooperative perception is a key and challenging task. This paper introduces a heterogeneous multiscale cooperative perception (HM-CoPept) framework with bird’s eye view features. It pays more attention to crucial cooperative data that affects driving to avoid blind areas and extend the sensing range. For the heterogeneity of V2X data, a bidirectional crossattention is innovatively proposed to fuse LiDAR and camera data complementary. Furthermore, the multiscale cooperation of V2X interaction data is proposed to break perception occlusion and limitation considering the benefit of multiscale features with different spatial importance. Cooperative perception is enhanced by learnable spatial confidence weight of safety distance constraint and foreground estimation. Test and validation are conducted on standard benchmarks, simulated (OPV2V) and real (DAIRV2X). HM-CoPept results show that performance increases by more than 16% compared to single-vehicle perception. Through extensive experiments and critical analysis, we demonstrate that our approach advances competitive methods and state-of-the-art in average precision. HM-CoPept enables precise and broad perception in complex driving environments and promotes the intelligent and autonomous development of CAVs.