Abstract 7439: Label-free 3D virtual papanicolaou staining of urine cytology using deep learning and holotomography

Abstract The Papanicolaou stain is commonly used to assess cells for carcinoma in urine cytopathology. However, due to the nature of cytology, cells are often imaged overlapping one another, which makes them hard to distinguish and complicates the assessment process. The staining procedure not only introduces changes to the cells, but it is also lengthy and expensive. Here, we present a deep learning model that creates virtual Papanicolaou stained 3D images from label-free 3D holotomography images of urine cytology. We acquired 38 3D holotomography images of urine cytology samples using the Tomocube HT-X1, after which the samples were stained and 38 matching whole slide images were acquired. The image pairs were preprocessed to flatten the 3D holotomography images into 2D, and image registration to align the image pairs. This produced 10, 693 image pairs of size 256x256 px, 8670 of which were used to train a modified version of the CycleGAN model. After the model was fully trained, the z-stack 3D holotomography images were input into the model one by one, resulting in a 3D virtual stained image. Our results show that our model was able to virtually stain the test sample (SSIM = 0.691 ± 0.097, PSNR = 19.9 ± 5.8, MSE = 0.017 ± 0.011) to produce an accurate 2D image, and also virtually visualize the Papanicolaou stained test samples in 3D. The presented work has potential in aiding practitioners by shortening the standard staining process. Through visualizing virtually stained urine cytology samples in 3D, our work provides the information needed to accurately assess cases of overlapping cells and cell clusters. Citation Format: Tal Lifshitz, Geon Kim, Su-Jin Shin, Kwang Suk Lee, YongKeun Park. Label-free 3D virtual papanicolaou staining of urine cytology using deep learning and holotomography [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 7439.