MDNet: Mamba-Effective Diffusion-Distillation Network for RGB-Thermal Urban Dense Prediction

In recent years, significant progress has been achieved in urban dense prediction tasks, particularly with advancements in deep learning models and novel architectures that enhance segmentation accuracy and computational efficiency. However, the following challenges persist: i) Existing modal fusion methods typically adopt convolutional neural networks (CNNs) or transformer (Trans)-based methods, which lead to inadequate global modeling or excessive computation owing to the introduction of quadratic complexity modeling; and ii) existing dense prediction networks typically utilize discriminative networks (codecs), which result in networks with insufficient discriminative properties. To address these issues, we propose the Mamba-effective diffusion-distillation network (MDNet) for RGB-thermal urban dense prediction. First, a new Mamba-effective fusion module is proposed, which efficiently models long-range pixel-level features using Mamba and generates pixel-level adaptive weights to fully utilize complementary modal information. Second, inspired by human self-reflection, a new diffusion self-distillation (DSD) strategy is proposed. The DSD generates coarse-grained binary semantic information via conditional multimodal image diffusion, which serves as self-distillation labels to improve the discriminative properties of the network. Experimental results demonstrate that the proposed MDNet achieves state-of-the-art performance on the MFNet dataset with fewer parameters and reduced computational effort. Extended experiments on the PST900 dataset further illustrate the effectiveness and generalizability of MDNet. The source code and results are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Tortoisewhp/MDNet</uri>.