Design of deep learning models for real-time gas leak monitoring

Nowadays, more and more people are concerned about the threats to human health and atmospheric pollution caused by gas leaks. The lack of efficient gas leak monitoring methods makes it difficult to address the problem effectively. Although some vision-based approaches attempt to monitor gas leaks in infrared videos, the transparent and non-rigid nature of gases often poses a challenge to these techniques. To address these problems, the thesis explores several feasible approaches and first proposes a Fine-Grained Spatial-Temporal Perception (FGSTP) algorithm for gas leak monitoring. FGSTP captures critical motion clues across frames and integrates them with refined object features. A correlation volume is first used to capture motion information between consecutive frames. Then, the spatial perception progressively refines the object-level features using previous outputs. Experiments demonstrate that the proposed model excels in segmenting non-rigid objects like gas leaks, generating the most accurate mask compared to other state-of-the-art (SOTA) models. However, sometimes the FGSTP is misled by objects that have similar motion or appearance, and it may also generate false positives on non-leak frames in a video because of the strong noise disturbance. Therefore, we modify FGSTP and design a novel model to solve those problems. The new network, Vision-Language Joint Gas Leak Segmentation (VLJGS), integrates the complementary strengths of video and text modalities to enhance the representation and detection of gas leaks. Additionally, the model employs a post-processing step to eliminate the false positives, ensuring that the model is not disturbed by non-target objects. Experiments demonstrate that the proposed model outperforms SOTA models. We evaluate the model using both supervised and few-shot training approaches, and the proposed model achieves excellent results in both cases, whereas existing models either perform well only in one scenario or poorly in both. In summary, this thesis presents two novel methods for gas leak monitoring, demonstrating the importance of multi-modality fusion. The findings of the thesis establish a foundation for future research on quantifying volumes or identifying types of gas leaks.