Feature Modulation and Aggregation Network for Efficient Image Super-resolution

With the increasing availability of hardware devices, deep learning-based super-resolution (SR) models have demonstrated extraordinary superiority compared to conventional hand-crafted methods. However, striking a delicate balance between effectiveness and efficiency remains a significant challenge for single image super-resolution (SISR) methods, particularly when deployed on resource-limited devices. In this paper, we propose a novel Feature Modulation and Aggregation Network (FMA-Net) for efficient image SR. The key components of our FMA-Net are the Efficient Feature Aggregation (EFA) module and Feature Refinement Convolution-based Feed-Forward Network (FRCFN). To collaboratively harness non-local interactions and local details, we employ EFA module to form the representative features. Additionally, we integrate FRCFN to further guide the network in focusing on more refined image attributes through a gating mechanism strategy. By coupling these two efficient modules, FMA-Net can capture both short-range and long-range contextual information for high-quality image SR. Our experiments on public benchmark datasets extensively demonstrate that FMA-Net achieves a favorable balance between computational efficiency and super-resolution performance, outperforming most lightweight baseline methods while maintaining reduced model complexity. In particular, compared to the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\times 4$</tex> ShufflerMixer, FMA-Net achieves an average improvement of 0.1dB across four public benchmark datasets while using only about 39.3% of the model complexity (e.g., FLOPs). Our source codes will be made available.