A Decoupled Segmentation-Classification Strategy Based on Semantic-SAM for Precise Semantic Segmentation in Coal Mine Areas

Addressing the critical challenges of complex semantic segmentation in coal mine areas, this study proposes a precise recognition method based on Semantic-SAM. We developed the SAM-SEF (SAM-based Semantic Enhancement Framework) framework, which innovatively integrates the zero-shot segmentation capability of Semantic-SAM with specialized deep learning models. Through a “segmentation-classification-update” strategy, accurate segmentation is achieved. The framework initially employs Semantic-SAM to obtain category-agnostic multi-granularity segmentation results, followed by semantic understanding of the segmented regions using specialized deep learning models, and finally generates precise segmentation results through a category mapping mechanism. Experiments evaluated 18 deep learning models, including ResNet series, Vision Transformer series, classical convolutional neural networks, and lightweight networks. Results indicate that the SAM-SEF-ENet-B0 model, based on the SAM-SEF framework, performs most prominently in segmenting Open-pit Coal Mining Sites (OCMS) and Composite Coal-related Sites (CCS), achieving IoUs of 93.68% and 81.52%, respectively, and an mIoU of 87.60%, significantly outperforming traditional segmentation models. Additionally, a semi-automatic annotation tool based on Semantic-SAM was developed, reducing traditional manual annotation time by approximately 60% through pre-segmentation and interactive editing functions. A multi-scale evaluation system based on the 33% and 66% percentiles of target area was established, revealing a clear positive correlation between target scale and segmentation performance, with small-scale target segmentation being the primary challenge (IoU of 0.79 for OCMS and 0.67 for CCS in small scales). Systematic experiments validated the necessity of selecting Semantic-SAM, showing that Semantic-SAM has significant advantages over SAM (mIoU of 54.03%) and SAM2 (mIoU of 45.33%) in semantic understanding capability, architectural adaptability, and computational efficiency. The study also found that classification networks based on the SAM-SEF framework achieved superior performance in segmentation tasks compared to specially designed segmentation networks, demonstrating the unique advantages of decoupling semantic segmentation tasks into “general segmentation + specialized classification” strategies in specific task domains. This research not only provides a new technical solution for remote sensing monitoring of coal mine areas but also offers valuable insights for the development of remote sensing image analysis, particularly demonstrating the effectiveness of a “classification-driven segmentation” paradigm where lightweight classifiers operating on purified targets from a frozen foundation model can outperform traditional end-to-end segmentation networks in specific, complex domains.