Research on turbulence-removal optical imaging based on multi-scale GAN and sequential images

Atmospheric turbulence severely degrades optical imaging quality by inducing significant geometric distortions and spatial blurring, posing a critical bottleneck in applications such as military reconnaissance and disaster monitoring. To address this challenge, we propose a novel Generative Adversarial Network (GAN), termed MS-TS-GAN (Multi-Scale Spatio-Temporal GAN), for effective turbulence mitigation. Our network innovatively integrates multi-scale convolutional structures and a spatio-temporal feature extraction mechanism. At the core of MS-TS-GAN is its generator, which uniquely combines a multi-scale convolutional structure to enhance the perception and capture of multi-scale turbulence information, ensuring comprehensive extraction of both global and fine-grained image features. Concurrently, a dedicated spatio-temporal feature extraction module is employed to accurately correct dynamic geometric distortions and motion blur induced by turbulence within image sequences. The discriminator, built upon a VGG network, integrates both global and local adversarial mechanisms, ensuring the visual fidelity and detail accuracy of the restored images through adversarial learning. Extensive experimental results, conducted on both numerical simulations and real-world turbulent scenes, demonstrate the superior performance and robustness of MS-TS-GAN across key metrics such as Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), Entropy (EN), and Average Gradient (AG). Our method significantly outperforms traditional approaches (e.g., Wiener filtering and inverse filtering) and other advanced GAN architectures (e.g., TS-GAN, SF-GAN). Specifically, under moderate turbulence, MS-TS-GAN achieves a PSNR of 18.6173 dB and an SSIM of 0.8059; for weak turbulence, it yields a PSNR of 18.7328 dB and an SSIM of 0.7858. Notably, compared to Wiener filtering, MS-TS-GAN demonstrates PSNR gains exceeding 36% and SSIM improvements surpassing 19% across both conditions. Furthermore, for no-reference natural scene images, our approach enhances EN by 0.1533 and AG by 0.0034 compared to Wiener filtering, indicating superior robustness. Ablation studies further confirm the crucial contributions of the multi-scale convolutional structure and the spatio-temporal feature extraction module to these performance enhancements. This study not only effectively corrects geometric distortions and spatial blurring but also significantly enhances image clarity and detail preservation. Our proposed MS-TS-GAN offers a novel, efficient, and robust solution for improving optical imaging system performance in turbulent environments, holding significant theoretical and practical value for overcoming current technical challenges.