Detecting beats in the photoplethysmogram: benchmarking open-source algorithms
The photoplethysmogram (PPG) signal is widely used in pulse oximeters and smartwatches. A fundamental step in analysing the PPG is the detection of heartbeats. Several PPG beat detection algorithms have been proposed, although it is not clear which performs best. Objective: This study aimed to: (i) develop a framework with which to design and test PPG beat detectors; (ii) assess the performance of PPG beat detectors in different use cases; and (iii) investigate how their performance is affected by patient demographics and physiology. Approach: Fifteen beat detectors were assessed against electrocardiogram-derived heartbeats using data from eight datasets. Performance was assessed using the F1 score, which combines sensitivity and positive predictive value. Main results: Eight beat detectors performed well in the absence of movement, with F1 scores of ≥90% on hospital data and wearable data collected at rest. Their performance was poorer during exercise, with F1 scores of 55-91%; poorer in neonates than adults with F1 scores of 84-96% in neonates compared to 98-99% in adults; and poorer in atrial fibrillation (AF), with F1 scores of 92-97% in AF, compared to 99-100% in normal sinus rhythm. Significance: Two PPG beat detectors denoted 'MSPTD' and 'qppg' performed best, with complementary performance characteristics. This evidence can be used to inform the choice of PPG beat detector algorithm. The algorithms, datasets, and assessment framework are freely available.
PDF Abstract
Datasets
MIMIC PERform Testing Dataset
MIMIC-III
Results from the Paper
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
|---|---|---|---|---|---|---|
| Photoplethysmography (PPG) heart rate estimation | MIMIC PERform Testing Dataset | MSPTD: MultiScale Peak & Trough Detection | MAPE | 2.4 | # 1 | |
| Photoplethysmography (PPG) heart rate estimation | MIMIC PERform Testing Dataset | SWT: Stationary Wavelet Transform | MAPE | 51.0 | # 15 | |
| Photoplethysmography (PPG) heart rate estimation | MIMIC PERform Testing Dataset | ATM: Adaptive Threshold Method | MAPE | 38.5 | # 14 | |
| Photoplethysmography (PPG) heart rate estimation | MIMIC PERform Testing Dataset | WFD: Wavelet Foot Delineation | MAPE | 17.5 | # 13 | |
| Photoplethysmography (PPG) heart rate estimation | MIMIC PERform Testing Dataset | PDA: Peak Detection Algorithm | MAPE | 7.8 | # 9 | |
| Photoplethysmography (PPG) heart rate estimation | MIMIC PERform Testing Dataset | COppg: Percentile Peak Detector | MAPE | 9.6 | # 12 | |
| Photoplethysmography (PPG) heart rate estimation | MIMIC PERform Testing Dataset | PWD: Pulse Wave Delineator | MAPE | 8.4 | # 11 | |
| Photoplethysmography (PPG) heart rate estimation | MIMIC PERform Testing Dataset | IMS: Incremental Merge Segmentation | MAPE | 7.7 | # 8 | |
| Photoplethysmography (PPG) heart rate estimation | MIMIC PERform Testing Dataset | ERMA: EventRelated Moving Averages | MAPE | 7.9 | # 10 | |
| Photoplethysmography (PPG) heart rate estimation | MIMIC PERform Testing Dataset | SPAR: Symmetric Projection Attractor Reconstruction | MAPE | 4.6 | # 6 | |
| Photoplethysmography (PPG) heart rate estimation | MIMIC PERform Testing Dataset | HeartPy | MAPE | 4.9 | # 7 | |
| Photoplethysmography (PPG) heart rate estimation | MIMIC PERform Testing Dataset | Pulses | MAPE | 3.3 | # 3 | |
| Photoplethysmography (PPG) heart rate estimation | MIMIC PERform Testing Dataset | ABD: Automatic Beat Detection | MAPE | 3.3 | # 3 | |
| Photoplethysmography (PPG) heart rate estimation | MIMIC PERform Testing Dataset | qppg: Adapted Onset Detector | MAPE | 3.5 | # 5 | |
| Photoplethysmography (PPG) heart rate estimation | MIMIC PERform Testing Dataset | AMPD: Automatic Multiscale Peak Detection | MAPE | 2.9 | # 2 | |
| Photoplethysmography (PPG) beat detection | MIMIC PERform Testing Dataset | SWT: Stationary Wavelet Transform | F1 score | 59.0 | # 15 | |
| Photoplethysmography (PPG) beat detection | MIMIC PERform Testing Dataset | ATM: Adaptive Threshold Method | F1 score | 71.1 | # 14 | |
| Photoplethysmography (PPG) beat detection | MIMIC PERform Testing Dataset | WFD: Wavelet Foot Delineation | F1 score | 86.5 | # 13 | |
| Photoplethysmography (PPG) beat detection | MIMIC PERform Testing Dataset | PDA: Peak Detection Algorithm | F1 score | 92.2 | # 12 | |
| Photoplethysmography (PPG) beat detection | MIMIC PERform Testing Dataset | COppg: Percentile Peak Detector | F1 score | 92.4 | # 11 | |
| Photoplethysmography (PPG) beat detection | MIMIC PERform Testing Dataset | PWD: Pulse Wave Delineator | F1 score | 92.9 | # 10 | |
| Photoplethysmography (PPG) beat detection | MIMIC PERform Testing Dataset | IMS: Incremental Merge Segmentation | F1 score | 93.6 | # 8 | |
| Photoplethysmography (PPG) beat detection | MIMIC PERform Testing Dataset | ERMA: EventRelated Moving Averages | F1 score | 93.6 | # 8 | |
| Photoplethysmography (PPG) beat detection | MIMIC PERform Testing Dataset | SPAR: Symmetric Projection Attractor Reconstruction | F1 score | 95.3 | # 7 | |
| Photoplethysmography (PPG) beat detection | MIMIC PERform Testing Dataset | HeartPy | F1 score | 95.6 | # 6 | |
| Photoplethysmography (PPG) beat detection | MIMIC PERform Testing Dataset | Pulses | F1 score | 96.6 | # 5 | |
| Photoplethysmography (PPG) beat detection | MIMIC PERform Testing Dataset | ABD: Automatic Beat Detection | F1 score | 96.8 | # 4 | |
| Photoplethysmography (PPG) beat detection | MIMIC PERform Testing Dataset | AMPD: Automatic Multiscale Peak Detection | F1 score | 97.2 | # 2 | |
| Photoplethysmography (PPG) beat detection | MIMIC PERform Testing Dataset | qppg: Adapted Onset Detector | F1 score | 96.9 | # 3 | |
| Photoplethysmography (PPG) beat detection | MIMIC PERform Testing Dataset | MSPTD: MultiScale Peak & Trough Detection | F1 score | 97.5 | # 1 |
