UniRiskP: Multimodal fusion-based in-hospital mortality prediction and phenotyping

Abstract Electronic health records (EHRs) integrate structured time-series data and clinical records, providing a comprehensive foundation for clinical risk prediction tasks such as in-hospital mortality (IHM) and phenotyping (PHE). However, existing studies still face several critical limitations, including insufficient modeling of irregularly sampled time-series data, lack of explicit cross-modal semantic alignment, and degraded performance under severe class imbalance, particularly for minority classes in both binary and multi-label settings. To address these challenges, we propose a task-aware multimodal risk prediction framework, called UniRiskP, which jointly models temporal irregularity, cross-modal alignment, and imbalance-aware learning. Specifically, UniRiskP incorporates a Unified Time-Driven Embedding (UTDE) module to explicitly model irregular sampling patterns and missing values in structured time-series data by leveraging inter-event time intervals. In parallel, semantic representations are extracted from radiology reports using ClinicalBERT. A maximum mean discrepancy (MMD)-based alignment strategy is employed to project heterogeneous modalities into a shared representation space, and a Dual-level Alignment-based Fusion (DeAF) module enables deep cross-modal interaction. To handle task-specific imbalance, UniRiskP adopts an optimal transport-based reweighting strategy for IHM and a label-aware threshold optimization mechanism for PHE. Experiments on the MIMIC-IV dataset demonstrate that UniRiskP consistently outperforms existing methods in terms of AUROC, AUPRC, F1, and Recall. Ablation studies further verify the effectiveness of irregular temporal modeling, cross-modal alignment, and imbalance-aware strategies. These results highlight the importance of jointly addressing temporal irregularity and semantic alignment in multimodal clinical risk prediction. These findings suggest that UniRiskP has the potential to support reliable ICU risk assessment from heterogeneous EHR data and will facilitate clinical decision-making in real-world settings.