SSD-MIR: Towards Medical Image Restoration With Structured Visual State Space Duality

In the field of Medical image processing, Transformers and CNNs bring filter-based local sensing and attention-based global modelling, which strengthened their dominant position. However, the scales required in different applications vary, and improper usage or selection on models can lead to performance decrease, resulting in more serious consequences in downstream tasks. The recent advanced Structured State Space Model has shown its potential as a baseline model, and is further proved to maintaining implicit duality with the attention mechanism, The corresponding model Mamba2 shows its long-range modelling ability with optimized long-term memory and linear computational complexity.In this paper, we propose SSD-MIR, a novel architecture leveraging the advantages of state space duality mechanism. SSD-MIR includes three parts: input projection and feature representation, deep feature transformation, output projection and feature inversion. With the three components sequentially connected to form our model, it is able to model long-range dependencies and learn the mapping between low-quality input and high-quality output simultaneously. In the main deep feature transformation part, depthwise convolution is used for additional feature mapping as an alternative of 1D causal convolution, and an optimized quad-direction scanning mechanism is implemented to form the overall 2D state space dual module. The corresponding wrapper module is built in MetaFormer style with the usage of residual connection and LayerNorm. We perform extensive experiments on multiple medical image restoration tasks, and the results are superior to the state-of-the-arts, demonstrating the strength and efficacy of our model.