MUM: Enhancing Medical Diagnostics Through Unpaired Multimodal Data Integration

Most AI healthcare applications are single-modality, focusing on tasks like electrocardiogram-based arrhythmia detection or echocardiogram interpretation for heart function. Yet, effective clinical decision-making often requires integrating diverse data sources, such as biosignals, imaging, audio, and lab reports. While multimodal integration could allow more accurate, personalized diagnostics, current approaches rely heavily on paired datasets, which are costly and challenging to curate. This limits applicability, despite the availability of unpaired data. We propose Multi-Uni-Modal (MUM), an efficient multimodal multitask learning framework that handles both paired and unpaired datasets, combining the diagnostic power of multimodal learning with the simplicity of training unimodal models. Extensive experiments show that MUM improves task performance by 2.7-4% while reducing learnable parameters by 53.5%. In an imbalanced heart murmur detection task, it achieved a 16.9% improvement, high-lighting its robustness to data imbalance and the potential of unpaired data in healthcare applications.