SAR Image Time Series for Land Cover Mapping via Sparse Local–Global Temporal Transformer Network

Due to the effective embedding of temporal information, time-series SAR images can acquire more abundant land covers information, thereby effectively enhancing the accuracy of land cover mapping. However, the significant introduction of temporal features greatly increases the redundancy of deep features. Simultaneously, capturing the global-local features of time-series SAR images effectively while considering temporal features has a significant influence on time-series SAR land cover mapping. Motivated by these, a sparse 3D Transformer network based on joint learning of temporal-spatial features, named <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S</u>parse <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</u>ocal-<underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</u>lobal <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">T</u>emporal <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">T</u>ransformer <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Net</u>work (SL-GT2Net), is proposed for SAR land cover mapping, which has the ability to learn sparse global-local temporal-spatial features of SAR land covers. To emphasize crucial spatiotemporal features and suppress noise features, we propose a Spatial-Temporal Channel Refinement Block (STCRB) to mitigate feature redundancy. Additionally, we develop novel and effective Sparse Multi-scale Local Window Multi-Head Self-Attention (SMLW-MHSA) and Sparse Multi-scale Global Window Multi-Head Self-Attention (SMGW-MHSA) blocks. Both incorporate parallel multi-scale window partitioning and Top-K sparsification strategies to robustly model discriminative spatiotemporal features of land covers with vast scale variations, while maintaining linear complexity with respect to the number of tokens. The experimental results on three challenge time-series SAR datasets demonstrate that our SL-GT2Net exhibits outstanding competitiveness compared to other related networks. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/RSIP-NJUPT/SL-GT2Net</uri>.