Toward robust automated cardiovascular arrhythmia detection using self-supervised learning and 1-dimensional vision transformers

Cardiovascular diseases are the leading cause of death worldwide. With electrocardiogram (ECG) machines becoming more accessible, passive monitoring for arrhythmia detection is now possible. This work highlights the importance of self-supervised learning in detecting arrhythmias by leveraging large-scale unlabelled ECG data to improve performance and reduce overfitting to class imbalance and noise. We propose Masked Patch Modelling (MPM) and use 8.2 million unlabelled ECGs for self-supervised pre-training, introducing PatchECG, a 1D Transformer model that can be fine-tuned for various ECG tasks. PatchECG achieves state-of-the-art results on standard datasets, including PTB-XL multi-label classification, and sets new benchmarks on the largest and highest-quality multi-label dataset to date. Compared to existing methods, PatchECG is five times more computationally efficient while increasing model capacity by a factor of 14. We also compare the 1D PatchECG model to a state-of-the-art 2D vision Transformer, HeartBEiT, and observe significantly higher performance. Finally, ablation studies reveal a 2% performance improvement in handling class imbalance, label noise, and over-parameterization. These findings demonstrate the potential of self-supervised learning in advancing automated arrhythmia detection.