HMFENet: Hierarchical Matching Guided Feature Enhancement Network for Few-Shot RGB-Thermal Urban Scene Segmentation

RGB-Thermal semantic segmentation provides reliable support for intelligent traffic perception systems, such as road safety monitoring and autonomous driving perception, by fusing visible and thermal imaging modalities under adverse weather conditions and low-light environments at night. However, the scarcity of multimodal data and the high cost of annotations severely limit the generalization capability of traditional models. To address the core demands of urban scene segmentation, we propose a Hierarchical Matching Guided Feature Enhancement Network (HMFENet) tailored for few-shot learning. It tackles two major challenges: 1) scale diversity of traffic objects (e.g., vehicles and pedestrians) under limited labeled data, which significantly degrades segmentation accuracy; 2) information redundancy across multimodal features, which undermines the enhancement effect of the thermal modality on traffic object segmentation. HMFENet employs a hierarchical dense matching mechanism to establish multi-scale and multi-level feature alignment between query images and support samples. Additionally, it incorporates a mutual information minimization constraint to optimize cross-modal complementarity, thereby enhancing segmentation robustness in complex urban scenes. Experiments on the urban scene dataset, Tokyo Multi-Spectral-<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4^{i}$ </tex-math></inline-formula> demonstrate that the proposed method achieves state-of-the-art results: an improvement of 5.9% and 9.4% in mean mIoU for critical traffic objects under 1-shot and 5-shot settings, respectively, compared to baseline models. Furthermore, the complementary effect of the thermal modality contributes to a 2.5% improvement under the 1-shot setting. The proposed method provides a feasible solution for deploying multimodal traffic perception systems with low annotation costs. The source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Zhou-xy99/HMFENet</uri>.