External validation of a machine learning based classification algorithm for ambulatory heart rhythm diagnostics using smartphone-photoplethysmography - the SMARTBEATS algorithm study

Abstract Introduction Smartphone-photoplethysmography (PPG) can be used for heart rhythm monitoring. According to current guidelines, ECG verification is required for a new diagnosis of atrial fibrillation (AF). Accurate machine learning (ML) based automatic algorithms have the potential to facilitate clinical applications generating large numbers of PPG recordings, by offloading the need for manual over-read. Purpose The aim of this study was to evaluate the diagnostic performance of an automatic ML-based algorithm for heart rhythm diagnostics using smartphone-PPG recorded by patients with AF in an unsupervised ambulatory setting. Methods Patients undergoing direct current cardioversion (DCCV) at a university hospital for treatment of AF or atrial flutter (AFL) were asked to perform one-minute heart rhythm recordings post-cardioversion at least twice daily for 30 days. All participants were provided with an iPhone 7 smartphone running the CORAI Heart Monitor PPG application simultaneously with a single-lead ECG recording (KardiaMobile). The automatic algorithm uses support vector machines (SVM) to classify heart rhythm from smartphone-PPG recordings. The SVM classifier evaluated here was trained on ambulatory PPG recordings made by patients in a separate cardioversion cohort, ensuring complete independence between training and validation sets. PPG recordings in the validation cohort were analyzed by the automatic algorithm and diagnostic performance was calculated by comparing heart rhythm classification output from the SVM classifier to the heart rhythm diagnosis from the simultaneous ECG recordings. ECG recordings were interpreted independently by two cardiologists and were set as gold standard. Results The SVM classifier was trained on data from 153 patients with 11,749 PPG recordings, of which 5,873 (50.0%) were labelled as AF and 5,876 (50.0%) as sinus rhythm (SR), as assessed by manual reading. In the validation cohort, 280 patients, with a median age of 69.0 years (31% women), registered 18,005 simultaneous PPG and ECG recordings. Recordings interpreted as AFL on ECG (2.0%), as having insufficient quality on ECG (4.9%) or PPG (2.8%), and low certainty algorithmic classifications (1.2%) were excluded, leaving 16,057 PPG recordings. Included ECG recordings consisted of 71.2% interpreted as (SR) and 28.8% as AF. Algorithm classification of the PPG recordings in the validation cohort diagnosed AF (sensitivity) in 99.7% and SR (specificity) in 99.7% of the recordings, with an overall accuracy of 99.7%. F1-score was 99.4% and area under the ROC curve (AUC) was 0.999. Conclusion A machine learning based algorithm for automatic heart rhythm diagnostics showed excellent diagnostic performance for smartphone-PPG recordings in an unsupervised ambulatory setting, minimizing the need for ECG verification in cardioversion populations.