A ViT-like unified multi-scale learning network for efficient image super-resolution

Recently, vision transformer (ViT)-based super-resolution (SR) models have achieved strong reconstruction performance but suffer from substantial computational and memory overhead due to self-attention operations, limiting their applicability in resource-constrained scenarios. Although several CNN-based alternatives attempt to replicate the global modeling capability of ViTs, their limited receptive fields often struggle to achieve competitive performance. To address this challenge, we propose a ViT-like unified multi-scale learning network (UMLN) that achieves a favorable trade-off between reconstruction quality and computational efficiency. Instead of relying on memory-intensive self-attention, our ViT-like design adopts a unified multi-scale feature aggregation mechanism that generates a global weight matrix through efficient convolutional operations. This unified module enables cross-scale interaction and long-range dependency modeling while significantly reducing parameter redundancy and memory consumption compared to attention-based approaches. In addition, we design an efficient feature extraction module CGDConv that effectively captures both local textures and non-local contextual information. Extensive experiments demonstrate that our UMLN outperforms existing efficient SR methods on public benchmark datasets, matching the performance of state-of-the-art lightweight ViT-based approaches while reducing significantly computational cost. Notably, compared to the × 4 SRFormer-light, our UMLN-L achieves an average PSNR improvement of 0.14 dB across five public benchmark datasets, while requiring only 68% of the computational complexity (e.g., FLOPs). Code and models are obtainable at https://github.com/hms-source/UMLN . • We propose a unified multi-scale feature extraction strategy that shares a single processing module across different scales, effectively reducing parameters and improving efficiency. • We propose a ViT-like weighted fusion mechanism to aggregate multi-scale features, enabling more effective global context modeling and overcoming the performance limitations of existing methods. • We design an effective feature extraction module, CGDConv, to capture both high- and low-frequency information within the network. • Extensive experiment results on public benchmark datasets demonstrate that the proposed UMLN achieves competitive performance in terms of model complexity, computational efficiency, and reconstruction quality compared to state-of-the-art approaches.