ASFANet: An Asymmetric Spatial-Frequency Sparse Attention Network for Image Deraining

Transformer-based methods have achieved remarkable progress in single-image deraining through long-range dependency modeling. However, they remain limited by redundant responses from dense channel interactions, insufficient characterization of rain-induced frequency degradation, and limited adaptability of U-shaped decoders to feature fusion and reconstruction. To address these issues, we propose ASFANet, an Asymmetric Spatial-Frequency Sparse Attention Network for image deraining. First, we develop Dynamic Sparse Channel Attention (DSCA) for adaptive feature selection. Its Spatial-Aware Predictor (SAP) adaptively estimates the proportion of channel responses to retain, while a Straight-Through Estimator (STE) provides surrogate gradients for hard top-k selection, enabling end-to-end optimization. Second, we introduce a Spatial-Frequency Feed-forward Network (SFFN) that combines spatially gated modeling and frequency modulation to capture local textures and rain-related frequency responses. Finally, we construct an asymmetric decoder with Upsampling Gated Fusion (UGF) and an Adaptive Dual-domain Gated Block (ADGB). It adaptively modulates and fuses skip features and performs spatial-frequency guided reconstruction to improve feature utilization and detail restoration. Experiments on four public benchmarks, including Rain200H, demonstrate superior deraining performance, with PSNR gains of 0.10–0.76 dB over Restormer. Further ablations show that the asymmetric decoder maintains restoration performance while reducing parameters by 4.55 M compared with the original homogeneous Transformer decoder.