Multiscale Structure Prompted Diffusion Models for Scene Text Image Super-Resolution

Scene Text Image Super-Resolution (STISR), which aims to enhance the resolution and legibility of text in low-resolution images, is crucial for accurate text recognition in real-world scenarios, especially in virtual reality (VR) environments where clarity is essential for immersive experiences. However, existing STISR methods often fall short of capturing detailed structural information, limiting the quality of reconstructed images. While diffusion models offer robust generative abilities, they can still result in structural inconsistencies, particularly when handling highly blurred or distorted text. To address these challenges, we introduce MSPDiff, a Multiscale Structure Prompted Diffusion Model designed specifically for STISR. MSPDiff integrates two main components: the Multiscale Structure Generation Model (MSGM) and the Prompted Diffusion Model (PDM). The MSGM captures multiscale structural features from text data, generating structural priors that guide the PDM. In this setup, the PDM leverages both the low-resolution images and the multiscale structures from the MSGM to enhance text clarity. Training both components simultaneously presents optimization challenges, as the MSGM is constrained by text data limitations while the PDM is optimized using noise. To address this, we implement a two-stage training approach: the MSGM is trained initially, after which the PDM is trained with the MSGM frozen. Experiments on the widely-used TextZoom benchmark demonstrate that MSPDiff outperforms state-of-the-art methods in both text recognition accuracy and image quality, establishing it as a promising approach for enhancing text clarity in virtual reality applications.