Multi-modal neural network for ECG signal classification in coronary artery disease using gramian angular field
Background Cardiovascular diseases (CVDs) remain a major global health burden, and electrocardiogram (ECG) classification is an important component of automated cardiac screening. However, conventional deep learning approaches may lose complementary information when operating on a single representation of the signal. This study aims to develop a multimodal method for normal-versus-abnormal ECG heartbeat classification under patient-independent evaluation. Methods We propose a multi-modal neural network that integrates raw ECG signals and their Gramian Angular Field (GAF) image representations. This dual-modal strategy enables the model to capture complementary time-series and two-dimensional structural features. The classifier combines lightweight multi-head attention with a residual feed-forward network, and the ECG, GASF, and fused representations provide coordinated prediction pathways followed by majority voting. To address generalization and patient-level leakage concerns, all formal experiments used strict patient-level five-fold cross-validation with patient-disjoint training, validation, and test partitions. Checkpoints were selected using validation ROC AUC, and the same fixed patient partitions were used for the ablation study and the BiLSTM/CNN + Transformer comparisons. Results Under the strict patient-level five-fold protocol, the Full Multimodal model achieved a ROC AUC of 0.996 and an accuracy (ACC) of 0.983. In the protocol-matched comparison, ECG-only achieved 0.815/0.846, BiLSTM 0.758/0.842, and CNN + Transformer 0.854/0.884 for ROC AUC/ACC, respectively. The ablation variants all showed lower ROC AUC and ACC than the complete model, supporting the contribution of the coordinated multimodal framework. Conclusion The proposed framework provides a patient-independent approach to normal-versus-abnormal ECG heartbeat classification by coordinating raw ECG and GASF representations. The revised patient-level cross-validation, component-level ablation, protocol-matched comparison, and updated literature context provide a more rigorous evaluation of the framework while supporting further validation in broader and external ECG cohorts.
Authors
- Jiayin Wang (ORCID: https://orcid.org/0000-0002-3862-6557)
- Ying Zhao (ORCID: https://orcid.org/0009-0005-5306-8134)
- Kai-qi Li
- Ting-miao Tang
- Jian-zhe Huang
- Yong Jiang
- Yan-ting Zhu
- Xin-hua Zou
- Xiang Pan
- Han-qi Yang
- Shi-yun Nie
Institutions
- Jiangnan University (CN)
- Jilin City Central Hospital (CN)
- Jilin Medical University (CN)
- Artificial Intelligence in Medicine (Canada) (CA)
Publication Details
- Journal
- Frontiers in Cardiovascular Medicine
- Published
- 2026-11-16
- DOI
- https://doi.org/10.3389/fcvm.2026.1844501
- Primary Topic
- ECG Monitoring and Analysis
- Type
- article
- Field-Weighted Citation Impact
- 0.00