DeepFissureNets-Infrared-Visible: Infrared visible image fusion for boosting mining-induced ground fissure semantic segmentation

High-intensive underground mining has caused severe ground fissures, resulting in environmental degradation. Consequently, prompt detection is crucial to mitigate their environmental impact. However, the accurate segmentation of fissures in complex and variable scenes of visible imagery is a challenging issue. Our method, DeepFissureNets-Infrared-Visible (DFN-IV), highlights the potential of incorporating visible images with infrared information for improved ground fissure segmentation. DFN-IV adopts a two-step process. First, a fusion network is trained with the dual adversarial learning strategy fuses infrared and visible imaging, providing an integrated representation of fissure targets that combines the structural information with the textual details. Second, the fused images are processed by a fine-tuned segmentation network, which lever-ages knowledge injection to learn the distinctive characteristics of fissure targets effectively. Furthermore, an infrared-visible ground fissure dataset (IVGF) is built from an aerial investigation of the Daliuta Coal Mine. Extensive experiments reveal that our approach provides superior accuracy over single-modality image strategies employed in five segmentation models. Notably, DeeplabV3+ tested with DFN-IV improves by 9.7% and 11.13% in pixel accuracy and Intersection over Union (IoU), respectively, compared to solely visible images. Moreover, our method surpasses six state-of-the-art image fusion methods, achieving a 5.28% improvement in pixel accuracy and a 1.57% increase in IoU, respectively, compared to the second-best effective method. In addition, ablation studies further validate the significance of the dual adversarial learning module and the integrated knowledge injection strategy. By leveraging DFN-IV, we aim to quantify the impacts of mining-induced ground fissures, facilitating the implementation of intelligent safety measures.