Diffusion-Driven Pansharpening via Conditional Multi-Stage Fusion

Multispectral pansharpening aims to produce high-resolution multispectral (HRMS) images by fusing a low-resolution multispectral (LRMS) image with a corresponding panchromatic (PAN) image. While the rise of diffusion models has marked significant progress in this field, existing methods often struggle by directly feeding the PAN and LRMS images into the network, an operation that fails to provide sufficient guidance for generating high-quality HRMS images. To tackle this problem, we propose a novel multi-stage diffusion-driven model for pansharpening (MSDP) in this paper. Our method involves a three-stage conditional fusion mechanism where effective cross-modal prior fusion is achieved through a novel sparse attention module, with dynamic integration of PAN image shearlet coefficients to achieve multi-scale frequency enhancement. These conditions are injected into the model network via two modulation modules, ensuring the exceptional preservation of both spatial sharpness and spectral fidelity. Furthermore, a lightweight physical model is developed to accelerate the training process. Experimental evaluations on pansharpening datasets show that our proposed method achieves superior results compared to recent state-of-the-art (SOTA) techniques, highlighting its potential to advance the field of pansharpening.