From laboratory to field: normal map-aided multimodal instance segmentation for blasting fragmentation analysis

• A novel multimodal instance segmentation network (AFRNet) is proposed. • AFRNet employs adaptive modality perception and multi-scale attention fusion mechanisms. • Normal maps significantly improve boundary detection and structural segmentation accuracy. • Particle size distribution is automatically computed and analyzed based on segmentation results. • The method achieves less than 10% deviation from manual particle size measurements on-site. Particle size analysis of rock fragments plays a crucial role in mining engineering. However, traditional non-contact image-based methods typically rely solely on RGB images, which are highly sensitive to illumination changes, shadow interference, and fragment texture. This reliance limits the accuracy and generalization capability of conventional approaches and often necessitates retraining when applied across different scenes. To address these issues, this study introduces normal maps and provides a detailed analysis of their advantages in representing rock fragment features. Furthermore, we propose a multimodal instance segmentation framework named Adaptive Feature Recombination Network (AFRNet). AFRNet incorporates a modality effectiveness perception mechanism to adaptively guide the fusion process while suppressing interference from unreliable modalities. In addition, it employs a multi-scale attention fusion module to fully exploit and utilize the strength of each modality. This study systematically compares three fusion strategies—data-level, feature-level, and decision-level—and conducts experiments under various modality combinations. Experimental results demonstrate that incorporating normal maps significantly improves segmentation accuracy and enhances model robustness in degraded environments such as low illumination and shadow interference. Moreover, the model trained in a laboratory environment is directly transferred, without retraining, to a practical particle size analysis task at an actual mining site in Nanjing. The resulting particle size distribution curves exhibit a deviation of less than 10% compared with manually labeled results, validating the proposed method’s zero-cost transferability and engineering applicability.