Meta-Swin: Lightweight Image Super-Resolution Swin Transformer for Metaverse Applications

This paper presents an improved model based on the Swin Transformer for image super-resolution and enhancement, specifically focusing on applications relevant to the metaverse. Image compression is crucial for efficient image and video transmission and storage. Yet, it often results in compression artifacts and loss of original details, degrading compressed images' perceptual quality and fidelity. While modern methods for image restoration have primarily relied on convolutional neural networks, there have been limited attempts to leverage Transformers, which excel in complex vision tasks. This study investigates the lightweight Meta-Swin transformer-based model to enhance SwinIR specifically for image super-resolution in compressed image scenarios. We address key challenges in training the vision transformer and original Swin transformers, such as unstable training, inconsistencies in resolution between pre-training and fine-tuning stages, and limitations imposed by the scarcity of labeled data. We perform experiments on three typical tasks: eliminating JPEG compression artifacts, traditional image enhancement, and super-resolution for compressed input images. The experimental findings indicate that our newly proposed approach, Meta-Swin, significantly enhances the training convergence and performance of the original model SwinIR. Moreover, Meta-Swin outperforms SwinIR and IPT regarding results across different representative tasks. The findings highlight the potential of the modified transformer network as a powerful tool for super-resolution of low-resolution and compressed images and reconstruction, particularly in scenarios involving compressed inputs. The proposed lightweight transformer architecture is a good candidate for metaverse technologies by boosting visual quality and creating a more realistic digital experience.