SR - TGAN: Smoke Removal with Temporal Generative Adversarial Models in Robot-assisted Surgery

The occurrence of smoke during endoscopic surgery hampers the ease of navigation and obstructs clear visibility, thereby presenting challenges and amplifying risks within surgical procedures. Current image processing-based smoke removal methods predominantly utilize machine learning approaches, specifically Generative Adversarial Networks (GAN). However, they encounter challenges in effectively preserving fine details and generating realistic images, a critical reason being the i.i.d assumption of the inputs. To tackle these issues, we present SR-TGAN: an innovative approach for Smoke Removal with the Temporal Generative Adversarial Network. Our model leverages the temporal dynamics inherent in surgical videos to significantly enhance the reconstructed images' quality. Specifically, SR-TGAN integrates sequential contextual information from closely preceding frames to effectively eliminate smoke, especially in regions where inferring the background is challenging such as in a highly occluded region. By comparing our SR- TGAN with the state-of-the-art DeSmoke-LAP, our method exhibits enhanced effectiveness in eliminating smoke from a dataset of 500 test images. Both visual inspection and quantitative metrics support this conclusion. In particular, the JNBM metric exhibits improvement from 1.37 (input images) to 1.49 (DeSmoke-LAP generated images) to 1.51 (SR-TGAN generated images), while FADE decreases from 0.737 to 0.360 to 0.346 for the corresponding image sets. The implications of this study are significant as they have the potential to reduce surgical risks, alleviate surgeons' workload by reducing the need to remove smoke physically, and enhance the precision of other computer vision algorithms utilized in live endoscopic surgeries. The code is available at https://github.com/XuMengyaAmy/SR-TGAN.