Synthetic H&E Generation from Highly Multiplixed Immunofluorescence Imaging using Deep-Learning

Haematoxylin and Eosin (H&E) staining and ImmunoFluo-rescence (IF) imaging are usually performed for tissue analysis. H&E staining is preferred in some downstream tasks but are time-consuming and can degrade tissue during washing. We propose a deep learning approach using a U-Net to generate synthetic H&E images from their IF markers counterpart. Since we consider this a deterministic mapping problem, we compare against the traditional GAN approach in literature, finding comparable quality yet reduced training costs. Using a colorectal cancer tissue dataset with 50+ markers from CODEX technology, we conducted an ablation study showing that DRAQ5 and Hoechst strongly influence synthetic H&E quality. Comparing subsets of markers, we assessed synthetic H&E images in downstream tasks like tissue segmentation, demonstrating the model's applicability and enabling explain-ability by quantifying marker relationships to H&E staining.