A dataset of multi-modal UAV remote sensing data for semantic segmentation of open-pit coal mines

<p indent="0mm">Open-pit coal mines are characterized by their large spatial scale, complex surface morphology, and highly variable dynamic elements. Traditional manual inspection and single-source monitoring methods suffer from low efficiency, multiple blind spots, and high operational risks, posing serious threats to production safety. Leveraging advanced information technologies to obtain timely, comprehensive, and accurate insights into the entire mining process is of great significance for safety operations. This study utilized unmanned aerial vehicles equipped with a l LiDAR system and a five-lens oblique cameras for data acquisition, resulting in a multi-modal remote sensing dataset that integrates digital orthophoto maps (DOMs), digital surface models (DSMs), LiDAR point clouds, and 3D oblique models. 2,992 image samples were extracted and annotated for semantic segmentation. Based on this, the first multi-modal semantic segmentation dataset for open-pit coal mines was constructed, with a data volume of approximately 12.5 GB. Through training and prediction using the U-Net, RandLA-Net, and IKD-Net models—representing the image, point cloud, and multimodal fusion modalities, respectively—experiments demonstrate that the multimodal fusion strategy significantly improves segmentation accuracy: 92.24% for the image modality (U-Net), 79.15% for the point cloud modality (RandLA-Net), and 94.9% for the multimodal fusion model (IKD-Net).This dataset provides a valuable benchmark for open-pit mining research and holds significant scientific and engineering potential for advancing unmanned, safe, and sustainable operations in open-pit mines.