Dual-Stream Spatially Aware Transformer for Remote Sensing Image Captioning

Remote sensing image captioning(RSIC) aims to generate semantically rich and syntactically accurate descriptions for remote sensing images. However, due to the complex spatial layouts, occlusions, and overlapping objects in such images, caption generation is often challenged by semantic ambiguity. To address these issues, we propose a novel <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Dual-Stream Spatially-Aware Transformer (DSAT)</b>,which explicitly models both global and local spatial relationships to enhance spatial understanding. Specifically, DSAT introduces a <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Dual-Stream Feature Interaction (DFI)</b> module that extracts grid-level global features and region-level object features, and further enhances their respective spatial dependencies through multi-branch convolution and a graph attention network. Additionally, we design a <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Spatially-Aware Attention (SAA)</b> mechanism that encodes relative spatial relationships into the Transformer, allowing the model to better capture object distribution patterns and geometric relationships. Extensive experiments conducted on three benchmark datasets, namely Sydney-Captions, UCM-Captions, and RSICD, highlight the superior performance of DSAT. The proposed method achieves impressive CIDEr scores of 338.59%, 450.93%, and 275.36% on these datasets, respectively, demonstrating its effectiveness in generating high-quality captions for remote sensing images.