A new QRS detector stress test combining temporal jitter and F-score (JF) reveals significant performance differences amongst popular detectors

QRS detection within an electrocardiogram (ECG) is the basis of virtually any further processing and any error caused by this detection will propagate to further processing stages. However, standard benchmarking procedures of QRS detectors are seriously flawed because they report almost always close to 100% accuracy for any QRS detection algorithm. This is due to the use of large temporal error margins and noise-free ECG databases which grossly overestimate their performance. The use of a large fixed error margin masks temporal jitter between detection and ground truth measurements. Here, we present a new performance measure (JF) which combines temporal jitter with the F-score, and also an ECG database with decreasing levels of signal to noise ratios based on noise generated from different tasks. Our new performance measure JF fully encompasses all the types of errors that can occur, equally weights them and provides a percentage value which allows direct comparison between QRS detection algorithms. In combination with the new noisy ECG database, the JF performance measure now varies between 50% and 100% for different detectors and signal to noise conditions thereby making it possible to find the best detector for an application.