Uncertainty estimation of deep learning models for atrial fibrillation detection from Holter recordings: A benchmark study

With the development of deep learning (DL)-based methods, automated atrial fibrillation (AF) detection from electrocardiograms (ECGs) has recently gained much attention. Although the performance of DL has been encouraging, the susceptibility of DL models to overfitting would benefit from the exploration of uncertainty quantification (UQ) to ensure safe integration into clinical practice. However, there has been limited exploration of UQ methods in the context of DL models for AF detection using Holter ECG recordings, and a comprehensive comparison of various UQ techniques remains absent. This study addressed this gap by introducing a benchmark study wherein 11 distinct UQ methods were rigorously evaluated and compared across three public Holter repositories: IRIDIA-AF, Long-Term AF, and MIT-BIH AF datasets. A residual DL model was used for the UQ methods, which is one of the most common architectures in this domain for its ability to capture complex patterns within ECG data. The findings revealed that batch-ensemble (BE) and packed-ensemble (PE) outperformed other UQ methods concerning both performance, as quantified by sensitivity, specificity and expected calibration error, and computational efficiency. In addition, when we implemented reject inference to discard ECG segments where the model confidence was not sufficiently high, BE and PE still showed to reject the least number of samples, while retaining the highest detection performance.