STGas: infrared video gas leakage detection based on spatio-temporal feature fusion

Abstract Detection of dangerous gas leakages based on infrared thermal imaging has widespread applications in industrial safety. However, existing detection methods often fail to adequately extract both the spatial and temporal features of leaking gases and struggle to satisfy real-time detection requirements. To address these problems, an infrared video-based gas feature extraction network named as STGas is proposed to integrate spatio-temporal features. To enhance the extraction and representation of temporal features, a cross-temporal difference feature fusion module (CTDFF), a cascaded channel aggregation module (CCA) and an improved RepViT module (RepViT-G) are introduced. Specifically, the CTDFF module fuses cross-time frame difference features for temporal feature extraction by leveraging adaptive interval self-attention. The CCA module captures fine-grained gas leakage details by performing hierarchical local feature extraction across different channels. The RepViT-G module incorporates a global attention mechanism into the original RepViT framework to enhance global feature representation. Experimental results on the insubstantial object detection dataset (IOD-Video) demonstrate that STGas achieves 43.16% mAP50, outperforming state-of-the-art feature extraction networks. Moreover, STGas achieves 29 FPS detection speeds, satisfying the requirements for real-time detection scenarios.