A radar-video fusion method for 3D object detection based on on-board edge computing in intelligent connected environments

In the context of next-generation networks, autonomous vehicles act as critical edge devices, demanding real-time and efficient 3D environmental perception. Addressing the low detection efficiency of current 3D object detection models when deployed at the edge, this paper proposes a 3D object detection model based on the deep fusion of image and point cloud data. Specifically, first, the projection matrix is utilized to achieve point—by—point matching between the RGB information of the image and the LiDAR point cloud. Subsequently, the point cloud is projected onto a plane perpendicular to the height direction to generate a Bird's eye view (BEV) with RGB information, thus completing the data—level fusion of image and point cloud data. Next, this paper proposes a BEV height channel design method. The point cloud is divided into multiple height intervals to construct three height channels, and the point cloud is weighted by the Gaussian function to enhance the features of the point cloud in the height direction and effectively alleviate the problem of information loss. Secondly, the backbone network is designed as two parallel dual—branch convolutional neural network models, which are respectively used to extract the BEV features of the point cloud and the image features. Then, based on the attention mechanism and projection transformation rules, an image and point cloud feature fusion module (IPFM) is designed to fuse at different depths of the model, realizing the multi—stage feature—level fusion of images and point clouds. Finally, experimental results on the KITTI dataset show that the proposed model significantly improves detection speed while ensuring accuracy, demonstrating its strong potential for efficient deployment on resource-constrained edge devices.