Multimodal data fusion with irregular PSA kinetics for automated prostate cancer grading

Prostate cancer (PCa) detection and accurate grading remain critical challenges in medical diagnostics. While deep learning has shown promise in medical image analysis, existing computer-aided diagnosis approaches primarily focus on image recognition, overlooking patient-relevant information. Additionally, current multimodal fusion approaches face a significant limitation in their inability to effectively integrate and analyze irregular time-series data, such as prostate-specific antigen (PSA) measurements alongside imaging data, particularly in cases where measurements are taken at inconsistent intervals. Here, we present a novel multimodal fusion framework that effectively combines imaging data with longitudinal patient information, including irregular PSA measurements, demographic data, and laboratory results. Our architecture employs a custom embedding technique to handle temporal sequences without requiring complex preprocessing or imputation steps. We evaluated our framework on a comprehensive dataset of prostate cancer patients from multiple clinical centers, encompassing both internal and external validation cohorts. The integration of temporal PSA information with imaging embeddings resulted in superior performance compared to traditional image-only approaches, demonstrating an improved area under the receiver operating characteristic curve (AUC) (0.843 vs. 0.808) for detecting clinically significant prostate cancer (csPCa). Our approach also achieved substantially more accurate prostate disease grading with a quadratic weighted kappa (0.645 vs. 0.557), validated on 630 cases from the same institution. The model demonstrated robust performance (AUC of 0.765) when evaluated on an external dataset comprising 419 cases from multiple European centers, utilizing 160 different MRI devices. When compared to experienced radiologists using PI-RADS scoring, our model showed higher sensitivity (74.5% vs.62.2%) at matched specificity (76.5%) while maintaining comparable performance (98.3% vs.98.1%) at high-sensitivity operating point. Our approach shows particular promise in reducing unnecessary biopsies while maintaining high detection sensitivity, suggesting significant potential as a clinical decision support tool.