Multimodal Deep Learning for ECG Heartbeat Classification with SHAP-Based Interpretability

Electrocardiogram (ECG) signals serve as a critical non-invasive diagnostic tool, providing valuable insights into the electrical activity of the heart and are widely used in diagnosing arrhythmias—abnormal heart rhythms that can lead to severe cardiovascular complications or even fatality if left undiagnosed. However, detecting arrhythmias manually from ECG data is a labor-intensive and time-consuming process, which limits scalability in real-time, resource-constrained environments and often requires trained professionals. In this paper, we propose a robust multimodal deep learning (DL) model for classification of five heartbeat arrhythmia classes defined by the AAMI EC57 standard from ECG signals. Our model leverages temporal, spectral and morphological features extracted from ECG signals using QRS complexes, Continuous Wavelet Transform (CWT), and Markov Transition Field (MTF) to improve generalization and diagnostic accuracy. Evaluated on the widely used MITBIH Arrhythmia Database, our model achieves a classification accuracy of 98.4%, demonstrating performance on par with or exceeding several recent state-of-the-art approaches. Additionally, an interpretability model is incorporated to explain the classification outcomes using SHAP (SHapley Additive exPlanations) values, highlighting the key features contributing to each prediction. The proposed system offers a promising solution for intelligent ECG analysis and real-time arrhythmia screening in clinical and wearable healthcare applications.