3D Information Perception of Earthworks Based on Machine Vision and LiDAR Fusion

The environment at earthwork construction sites is complex, which severely restricts the development of autonomous operation of intelligent construction machinery. To address this issue, a perception method based on the fusion of machine vision and LiDAR is proposed for sensing earthwork information at construction sites. By leveraging the rich semantic information in camera images and the accurate three-dimensional information in LiDAR point clouds, earthwork targets in the images are extracted using the Mask R-CNN instance segmentation network model. Then, the three-dimensional point clouds from the LiDAR are projected onto the two-dimensional image to extract earthwork color point clouds, achieving the fusion perception of earthwork information. Experimental results indicate that this method enables efficient fusion of earthwork image data and point cloud data, achieving extraction of earthwork operation targets and three-dimensional information perception.