Multimodal Sensor Fusion for Road Surface Identification Considering Vehicle Dynamic Characteristics

Multi-source sensors, such as LiDAR, cameras, Inertial Measurement Units (IMU), and suspension displacement sensors, can describe road surface characteristics from different dimensions. Sensor fusion, which incorporates vehicle dynamic characteristics, is a key to improving the accuracy of road surface identification. Therefore, we propose a road surface identification method that combines segmented image features, statistically analyzed and extracted LiDAR features, and vehicle state features, with suspension displacement serving as a supervisory signal. The visual features from cameras and LiDAR inputs are extracted using the Transfuser backbone. Meanwhile, vehicle state features are encoded separately using Fourier Feature Mapping and a Multilayer Perceptron (MLP), and are subsequently fused with the visual features. The accuracy of road surface identification is further improved through the use of a state feature dropout module and a suspension displacement supervision module during the training process. Experimental results show that our method effectively combines multi-source sensor information and achieves higher accuracy in road surface identification compared to single-sensor-based methods and other multi-sensor fusion approaches. Furthermore, comparative road surface identification tests under constant and variable vehicle speeds, conducted under the same road conditions, demonstrate that our method is not affected by the type of vehicle motion, due to the supervision module based on the decoupled suspension displacement signal.