Multi-modal neural network for ECG signal classification in coronary artery disease using gramian angular field

Background Cardiovascular diseases (CVDs) remain a major global health burden, and electrocardiogram (ECG) classification is an important component of automated cardiac screening. However, conventional deep learning approaches may lose complementary information when operating on a single representation of the signal. This study aims to develop a multimodal method for normal-versus-abnormal ECG heartbeat classification under patient-independent evaluation. Methods We propose a multi-modal neural network that integrates raw ECG signals and their Gramian Angular Field (GAF) image representations. This dual-modal strategy enables the model to capture complementary time-series and two-dimensional structural features. The classifier combines lightweight multi-head attention with a residual feed-forward network, and the ECG, GASF, and fused representations provide coordinated prediction pathways followed by majority voting. To address generalization and patient-level leakage concerns, all formal experiments used strict patient-level five-fold cross-validation with patient-disjoint training, validation, and test partitions. Checkpoints were selected using validation ROC AUC, and the same fixed patient partitions were used for the ablation study and the BiLSTM/CNN + Transformer comparisons. Results Under the strict patient-level five-fold protocol, the Full Multimodal model achieved a ROC AUC of 0.996 and an accuracy (ACC) of 0.983. In the protocol-matched comparison, ECG-only achieved 0.815/0.846, BiLSTM 0.758/0.842, and CNN + Transformer 0.854/0.884 for ROC AUC/ACC, respectively. The ablation variants all showed lower ROC AUC and ACC than the complete model, supporting the contribution of the coordinated multimodal framework. Conclusion The proposed framework provides a patient-independent approach to normal-versus-abnormal ECG heartbeat classification by coordinating raw ECG and GASF representations. The revised patient-level cross-validation, component-level ablation, protocol-matched comparison, and updated literature context provide a more rigorous evaluation of the framework while supporting further validation in broader and external ECG cohorts.

Authors

Institutions

Publication Details

Journal
Frontiers in Cardiovascular Medicine
Published
2026-11-16
DOI
https://doi.org/10.3389/fcvm.2026.1844501
Primary Topic
ECG Monitoring and Analysis
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Multi-modal neural network for ECG signal classification in coronary artery disease using gramian angular field

Jiayin Wang, Ying Zhao, Kai-qi Li, Ting-miao Tang et al.
2 links
Frontiers in Cardiovascular Medicine
ECG Monitoring and Analysis
article

Multi-modal neural network for ECG signal classification in coronary artery disease using gramian angular field

Jiayin Wang, Ying Zhao, Kai-qi Li, Ting-miao Tang, Jian-zhe Huang, Yong Jiang, Yan-ting Zhu, Xin-hua Zou, Xiang Pan, Han-qi Yang, Shi-yun Nie
article en

Abstract

Background Cardiovascular diseases (CVDs) remain a major global health burden, and electrocardiogram (ECG) classification is an important component of automated cardiac screening. However, conventional deep learning approaches may lose complementary information when operating on a single representation of the signal. This study aims to develop a multimodal method for normal-versus-abnormal ECG heartbeat classification under patient-independent evaluation. Methods We propose a multi-modal neural network that integrates raw ECG signals and their Gramian Angular Field (GAF) image representations. This dual-modal strategy enables the model to capture complementary time-series and two-dimensional structural features. The classifier combines lightweight multi-head attention with a residual feed-forward network, and the ECG, GASF, and fused representations provide coordinated prediction pathways followed by majority voting. To address generalization and patient-level leakage concerns, all formal experiments used strict patient-level five-fold cross-validation with patient-disjoint training, validation, and test partitions. Checkpoints were selected using validation ROC AUC, and the same fixed patient partitions were used for the ablation study and the BiLSTM/CNN + Transformer comparisons. Results Under the strict patient-level five-fold protocol, the Full Multimodal model achieved a ROC AUC of 0.996 and an accuracy (ACC) of 0.983. In the protocol-matched comparison, ECG-only achieved 0.815/0.846, BiLSTM 0.758/0.842, and CNN + Transformer 0.854/0.884 for ROC AUC/ACC, respectively. The ablation variants all showed lower ROC AUC and ACC than the complete model, supporting the contribution of the coordinated multimodal framework. Conclusion The proposed framework provides a patient-independent approach to normal-versus-abnormal ECG heartbeat classification by coordinating raw ECG and GASF representations. The revised patient-level cross-validation, component-level ablation, protocol-matched comparison, and updated literature context provide a more rigorous evaluation of the framework while supporting further validation in broader and external ECG cohorts.

Frontiers in Cardiovascular MedicineVol. 13
Jiangnan University (CN), Jilin City Central Hospital (CN), Jilin Medical University (CN), Artificial Intelligence in Medicine (Canada) (CA)
Good health and well-being
ECG Monitoring and Analysis
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon