All-in-One Single Image Restoration Based on Multi-Scale Hybrid Mamba-Transformer

Aiming at the pain points of weak generalization and poor scene adaptability of existing methods in the field of image restoration, this paper proposes an integrated image restoration model named SIR-MSHMT based on the multi-scale hybrid Mamba-Transformer, which aims to be able to perform image restoration tasks in complex environments of different scenes. The model introduces a dual-branch MambaTransformer interaction module (DB-MTIB) and a MambaTransformer prompt module (MTPB). The interaction module contains a dynamic dual-channel attention (DDCA) and a serial bidirectional state space model (SBSM). The serial bidirectional state space model module processes the features in the onedimensional spatial direction, captures the dependencies of spatial positions, and efficiently models the spatial information. The dynamic dual-channel attention mechanism, through dynamic and adaptive masking operations, aims to screen and retain the most valuable attention values, while effectively eliminating interference noise, achieving multi-dimensional degradation perception and feature decoupling. In the prompt module, a learnable degradation embedding is used to dynamically adapt to complex degradation scenes, enhancing the content adaptability of the model. SIR-MSHMT performs excellently on multiple benchmark datasets and outperforms the current state-of-the-art integrated image restoration methods. Ablation experiments further verify the significant contributions of the dual-branch Mamba-Transformer interaction module and the Mamba-Transformer prompt module to the performance of the model. SIR-MSHMT utilizes a unified framework to solve the problem of restoring mixed degradation in real-world scenarios, breaking through the performance bottleneck of traditional taskspecific models.