Multi-Modal Data Fusion for Predictive Analytics in Healthcare: A Hierarchical Attention Framework for Early Sepsis Prediction

The integration of multi-modal healthcare data—encompassing electronic health records (EHRs), medical imaging, and unstructured clinical notes—holds transformative potential for predictive analytics but remains underexplored due to challenges in data heterogeneity and interpretability. This paper introduces a hierarchical attention-based fusion framework to predict sepsis onset 6 hours prior to clinical diagnosis, leveraging temporal (EHRs), spatial (chest X-rays), and textual (radiology notes) data. The proposed model employs modality-specific encoders: a bidirectional LSTM (BiLSTM) for EHR time-series, a ResNet-50 pretrained on chest X-rays, and a BioBERT transformer for clinical text. A novel cross-modal attention mechanism dynamically aligns features across modalities, addressing missing data and sampling rate disparities. Evaluated on a multi-center ICU dataset of 12,000 patients, the framework achieves an AUC-ROC of 94.3% (18% improvement over EHR-only models) and reduces false negatives by 22%, demonstrating statistically significant improvements (p < 0.001) in early sepsis detection. Clinical simulations reveal a 14% reduction in ICU mortality through timely interventions, validated by physician feedback. This work advances multi-modal fusion strategies for high-stakes clinical decision-making while maintaining interpretability via SHAP-based explainability.