Multi-Modal Integration for Predicting Biochemical Recurrence in Prostate Cancer

Predicting biochemical recurrence (BCR) is crucial for Prostate Cancer (PCa) treatment plan. We present a novel multi-modal approach using deep learning framework that integrates Whole Slide Images (WSI) with associated clinical text to improve BCR prediction. Our model leverages contrastive learning approach where both visual and textual data are used to effectively capture the interplay between histopathology features and textual information by jointly modeling these data sources. Evaluation results indicate that our model building approach is able to effectively integrates spatial insights from WSIs and clinical texts, leading in enhanced BCR prediction performance compared to other existing methodologies and individual modalities. This approach has the potential to advance precision medicine in PCa care.