Off-Road Terrain Surface Type Recognition Based on Multi-Sensor Information Fusion

This study aims to detect the surface type of the road ahead and the distance between the vehicle and the changing road surface in an off-road environment. The goal is achieved by fusing image and point cloud data. The specific process is as follows: acquire camera images and radar point cloud data, segment the images using the DeepLabv3+ network, and perform multi-frame point cloud fusion. The fused data is then transformed through coordinate conversion to assign color information to the point cloud. The 3D point cloud is flattened, and a custom algorithm is used to detect the road surface type. The experiment shows that the DeepLabv3+ model performs excellently across various evaluation metrics. In the multi-frame point cloud fusion, increasing the number of frames results in higher point cloud density, but issues such as point cloud overlap and longer algorithm execution time arise. A fusion of 2 frames is selected as the optimal approach. Ultimately, the experimental system was successfully built, effectively detecting the road surface type.