Unsupervised Diffusion-trajectory Segmentation and Multiregion Fusion for Understanding RGB–thermal Scene

Multispectral scene understanding remains a critical sensing challenge: visible-spectrum (RGB) image sensors provide high-resolution color and texture cues but fail under poor illumination, while thermal infrared sensors operating on the principle of detecting heat radiation emitted by objects are inherently robust to low-light conditions yet deliver lower spatial resolution and less distinct contrast characteristics.The effective fusion of these two sensor modalities is therefore essential for reliable autonomous perception in real-world environments.To address the sensing limitations of individual modalities and the scarcity of large, densely annotated RGB-thermal (RGB-T) datasets, we propose a hierarchical, multistage RGB-T framework that (1) performs adaptive layer-wise fusion of multiscale features from a dual-stream backbone (ResNet-18 for RGB sensor data, and a custom convolutional neural network with squeeze-and-excitation blocks for thermal sensor data), (2) produces sharp, unsupervised region segmentations by mining latent diffusion trajectories of a pretrained diffusion model, and (3) aggregates region features via a graph-structured hierarchical fusion network for robust scene reasoning.The proposed sensing pipeline preserves complementary local and global cues from both modalities, yields semantically coherent region proposals without manual annotations, and models region-to-region context for stronger scene descriptors.Evaluated on RGB-T benchmarks, the integrated pipeline improves segmentation (Mean Intersection over Union ≈ 0.758) and scene classification (mean accuracy ≈ 88%) over representative baselines, demonstrating superior boundary precision, object separation, and robustness to clutter and occlusion, highlighting its value for sensor-driven autonomous scene understanding systems.