RGB-LiDAR sensor fusion for dust de-filtering in autonomous excavation applications

The dusty environments of autonomous excavation can affect the performance of the sensors onboard the vehicle. Specifically, airborne dust clouds can be perceived as solid objects if not addressed appropriately, which can lead to irrational movements that risk safety. In this article, a light detection and ranging (LiDAR) and red-green-blue (RGB) image sensor fusion model was developed to filter airborne dust particles. The proposed approach processes the RGB and LiDAR data in separate convolutional neural network (CNN) models and combines the predictions in a late fusion model for enhanced real-time performance. Testing shows that the proposed fusion model has an F1 score at least 2.64% higher than a LiDAR only CNN model and a dynamic radius outlier removal paired with low-intensity outlier removal (LIOR-DROR) when dust clouds are around 3 m from the sensors. • Multi-layer deep learning approach for airborne dust particle filtering using late sensor fusion technology. • 2D RGB and 3D LiDAR data was fused using a projection scheme. • Optimal results were obtained when airborne dust is location approximately 3 m from the sensors. • Real-time performance was achieved with respect to the refresh rate of the LiDAR.