3D Rendering of Cystoscopy Video Footage: A Novel Method Utilizing Neural Radiance Field Processing

Recent developments in neural radiance field (NeRF) processing have leveraged the power of neural networks to quickly reconstruct 3D spaces from 2D images. Our objective was to utilize this technology to 3D render video recordings of diagnostic cystoscopies and test their fidelity. With institutional review board (IRB) approval, we recorded two diagnostic cystoscopies, one with an Ambu single-use flexible cystoscope and the other with a Richard Wolf digital cystoscope. We converted the videos to images and manually curated approximately 100 representative images, which minimized blur and spanned a large segment of the bladder. We then utilized the NVIDIA Instant Neural Graphics Primitives (iNGP), a NeRF algorithm that uses multiresolution hash encoding with a compact neural network for significantly faster convergence, to reconstruct the bladder and render novel, unseen views within seconds. We computed the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) to assess the quality and fidelity of the 3D rendering. Both videos were able to be utilized for 3D rendering using iNGP. The rendering derived from the Richard Wolf cystoscopy had a PSNR = 29.8 (min = 27.2, max = 32.6) and SSIM = 0.89. Similarly, the rendering derived from the Ambu cystoscopy had a PSNR = 31.3 (min = 27.1, max = 35.1) and SSIM = 0.90. Independent of cystoscopy equipment, both 3D renderings achieved reasonable fidelity. Major limitations to widespread adoption of this technology include the need for a curator to select representative and high-quality images from the initial cystoscopy video recording and the relatively small segments of bladder successfully rendered. Nonetheless, we feel that with further refinement, this technology can be scaled to create 3D renderings of cystoscopies that will enable evaluation of both completeness and quality of the cystoscopy. Furthermore, this technology would be able to facilitate the comparison of cystoscopies performed in the same patient over time.