Towards Accurate and Reliable ICU Outcome Prediction: A Multimodal Learning Framework Based on Belief Function Theory using Structured EHRs and Free-Text Notes

Abstract Accurate Intensive Care Unit (ICU) outcome prediction is critical for improving patient treatment quality and ICU resource allocation. Existing research mainly focuses on structured data, e.g. demographics and vital signs, and lacks effective frameworks to integrate clinical notes from heterogeneous electronic health records (EHRs). This study aims to explore a multimodal framework based on belief function theory that can effectively fuse heterogeneous structured EHRs and free-text notes for accurate and reliable ICU outcome prediction. The fusion strategy accounts for prediction uncertainty within each modality and conflicts between multimodal data. Experiments on two large ICU datasets demonstrate that our method achieves superior predictive performance compared to existing approaches. For example, it improving F1 score and AUPRC by 6.51% and 3.72%, respectively, and increases predictive reliability with an 18.08% decrease in Brier score for mortality prediction in MIMIC-III dataset. Comparable improvements are consistently observed on the ZICIP dataset, underscoring the predictability and reliability of the approach. These improvements translate into fewer false positives, supporting more precise triage decisions and more efficient allocation of critical care resources. Beyond ICU outcome prediction, the proposed framework offers a versatile tool for multimodal EHR analysis, with potential applications across diverse clinical tasks.