Multimodal fusion of ECG and chest X-ray with deep learning and radiomics for cardiac diagnosis: A machine learning-based nomogram approach

To develop and validate a comprehensive, multimodal diagnostic framework combining ECG, chest X-ray imaging, radiomic features, heart rate variability (HRV), and clinical data for accurate classification of major cardiac conditions. We enrolled 2285 patients from seven clinical centers, ultimately including 1987 for training and 498 for external testing. ECG signals were converted into standardized 2D grayscale images and processed using Vision Transformer (ViT) and EfficientNet models to extract deep features. Chest X-ray images underwent preprocessing, segmentation by expert radiologists, and feature extraction using the same deep learning architectures, supplemented by handcrafted radiomic analysis. All features underwent intraclass correlation coefficient (ICC) filtering (threshold ≥0.75) to ensure reliability. Dimensionality reduction and feature selection were performed using PCA and LASSO. The resultant features were fused with clinical variables (e.g., age, BMI, blood pressure, diabetes) and HRV metrics. Several machine learning classifiers were trained and validated. Interpretability was assessed using SHAP analysis, and a nomogram was developed for clinical decision-making. Decision curve analysis (DCA) was applied to evaluate clinical utility across risk thresholds. The integrated model achieved a training accuracy of 96.0 % and external test accuracy of 94.0 %, with an AUC of 0.95. Fused ECG features outperformed single-model inputs (AUC = 0.90), and radiomic features from chest X-ray provided robust discriminatory power (AUC = 0.86). SHAP analysis revealed distinct feature contributions across cardiac categories, and DCA showed superior net clinical benefit of the XGBoost model. Nomograms enabled individualized risk assessment. This study demonstrates that integrating multimodal data enhances cardiac disease classification accuracy and interpretability. The proposed framework is a scalable, reproducible solution for real-world clinical deployment.