A Novel Spectral-Spatial Attention Network for Zero-Shot Pansharpening

Over the past decades, pansharpening technologies have received much attention due to the spatial detail enhancement they introduce into multispectral (MS) images by referencing them to panchromatic images (PAN). Traditional approaches struggle to capture nonlinear relationships between the original MS-PAN pair and the pansharpened MS, whilst deep learning (DL)-based methods introduce new challenges. Unsupervised pansharpening is prone to distortion due to the unavailability of reference images and errors in modeling the degradation process. Supervised methods are trained at reduced resolutions before migrating to full resolution, which lead to undesirable results due to scale variations. Meanwhile, the constructed training data cannot encompass all possible scenes, hindering the reconstruction capability of unknown scenes. To tackle these challenges, we propose a novel zero-shot Pansharpening Network (<monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ZSPNet</monospace>), which exclusively conducts training and testing on the target image pair, ensuring the robust performance in unknown scenes. Furthermore, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ZSPNet</monospace> method can effectively reduce the scale variance due to its tailored network using an appropriate patch size. As a result, by integrating 3-D convolutional neural networks (3DCNN), spatial attention and channel attention, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ZSPNet</monospace> is capable of accurately reconstructing MS with enhanced spatial resolution. The experiments were conducted on the public dataset PAirMax, which have challenging scenes captured by different sensors. Compared to some state-of-the-art traditional and DL-based methods, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ZSPNet</monospace> demonstrates superior performance in both quantitative assessments and visual results.