DeepVMIF: Deep Learning Based Volumetric Multiplex Immunofluorescence Image Analysis

Three-dimensional (3D) pathological analyses of non-destructive whole biopsies and surgical specimens reveal substantial improvements in diagnostic performance. Advances in light sheet microscopy and tissue clearing techniques enable high-throughput examination of a significantly larger volume of specimens compared to traditional histology. Volume imaging of multiple rounds on thick sections cleared from tissue samples, with multiplex immunofluorescence labeling of biomarkers, enables the segmentation of these markers and facilitates the intuitive analysis of spatial correlations among multiple biomarkers in 3D space. However, the deformation of thick sections during multiple rounds of imaging is inevitable and poses challenges in data registration, marker segmentation, and co-expression detection. Herein, we present a highly efficient deep learning-based tool for volumetric multiplex immunofluorescence (Deep-VMIF) image analysis to address the aforementioned difficulties for 3D pathology. For biomarker segmentation, we developed a semi-supervised segmentation architecture with feature layer adversarial training, which achieves a Dice score above 0.86 when trained with only 20% labeled data, demonstrating exceptional performance. To solve the issue of multi-round 3D image registration, an alignment network and an iterative multi-layer pyramid registration network are designed. The network can register the 3D image with two rounds of normalized cross correlation (NCC) coefficient greater than 0.92 on test data. Finally, we evaluated the performance of the entire process on two-round three-channel 3D imaging data, which only took 75 minutes. DeepVMIF is a highly efficient and convenient tool for multiplexed 3D pathological image analysis of large specimens.