Confidence Does Not Travel: Cross-Dataset Validation of Selective AI for Photoplethysmographic Respiratory-Rate Monitoring Before Breathing-Centered Rehabilitation Deployment

Background/Objectives: Photoplethysmography (PPG) could support unobtrusive respiratory-rate monitoring during breathing-centered rehabilitation. Accuracy measured in one cohort, however, says little about how the same estimator will behave after deployment elsewhere. Confidence estimates may also fail. We tested whether physiology-informed artificial intelligence and a learned error-prediction gate retained their performance across two clinically distinct public datasets. Methods: We conducted a retrospective, subject-separated benchmark using 53 BIDMC intensive-care recordings and 42 CapnoBase anesthesia recordings. From 32 s PPG windows advanced every 8 s, we derived four respiratory modulations and 51 rate, agreement, morphology, and signal-quality features. Six estimators were evaluated by five-fold grouped cross-validation and by bidirectional external transfer. A nested gradient-boosting model predicted each estimate's absolute error to support selective reporting. Performance was summarized with subject-bootstrap confidence intervals, Bland–Altman agreement, and risk–coverage analysis. Results: The analysis included 5262 windows from 95 participants. Within-dataset mean absolute error (MAE) was lowest for random forest on BIDMC (1.65 breaths/min) and Extra Trees on CapnoBase (3.55 breaths/min). The physiology-consensus model achieved 1.70 and 4.02 breaths/min, respectively. Its learned gate reduced MAE at 50% coverage to 0.93 and 2.01 breaths/min. Under external transfer, however, its MAE increased to 6.19 breaths/min for BIDMC→CapnoBase and 4.90 breaths/min for CapnoBase→BIDMC. Uncertainty–error Spearman correlation fell from 0.48 and 0.41 internally to 0.15 and 0.09 externally, and gating yielded only modest improvement. A fixed raw-spectral baseline outperformed the learned model in both transfer directions. Conclusions: Confidence measured within the development cohort did not establish safety after transfer. PPG-derived respiratory rate needs external calibration before it can guide rehabilitation feedback, even when point accuracy appears acceptable internally. Public benchmarks should report bidirectional transfer and risk–coverage behavior alongside random or subject-wise internal splits.

Authors

Publication Details

Journal
Preprints.org
Published
2026-11-12
DOI
https://doi.org/10.20944/preprints202611.1249.v1
Primary Topic
Non-Invasive Vital Sign Monitoring
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Confidence Does Not Travel: Cross-Dataset Validation of Selective AI for Photoplethysmographic Respiratory-Rate Monitoring Before Breathing-Centered Rehabilitation Deployment

Ashif Newaz, Sayor Ajfar Aaron, Sajjat Hossain Abir, Nazmul Ahosan et al.
Preprints.org
Non-Invasive Vital Sign Monitoring
preprint

Confidence Does Not Travel: Cross-Dataset Validation of Selective AI for Photoplethysmographic Respiratory-Rate Monitoring Before Breathing-Centered Rehabilitation Deployment

Ashif Newaz, Sayor Ajfar Aaron, Sajjat Hossain Abir, Nazmul Ahosan, Md. Mushfiqur Rahman, Md. Monir Hossain
preprint en

Abstract

Background/Objectives: Photoplethysmography (PPG) could support unobtrusive respiratory-rate monitoring during breathing-centered rehabilitation. Accuracy measured in one cohort, however, says little about how the same estimator will behave after deployment elsewhere. Confidence estimates may also fail. We tested whether physiology-informed artificial intelligence and a learned error-prediction gate retained their performance across two clinically distinct public datasets. Methods: We conducted a retrospective, subject-separated benchmark using 53 BIDMC intensive-care recordings and 42 CapnoBase anesthesia recordings. From 32 s PPG windows advanced every 8 s, we derived four respiratory modulations and 51 rate, agreement, morphology, and signal-quality features. Six estimators were evaluated by five-fold grouped cross-validation and by bidirectional external transfer. A nested gradient-boosting model predicted each estimate's absolute error to support selective reporting. Performance was summarized with subject-bootstrap confidence intervals, Bland–Altman agreement, and risk–coverage analysis. Results: The analysis included 5262 windows from 95 participants. Within-dataset mean absolute error (MAE) was lowest for random forest on BIDMC (1.65 breaths/min) and Extra Trees on CapnoBase (3.55 breaths/min). The physiology-consensus model achieved 1.70 and 4.02 breaths/min, respectively. Its learned gate reduced MAE at 50% coverage to 0.93 and 2.01 breaths/min. Under external transfer, however, its MAE increased to 6.19 breaths/min for BIDMC→CapnoBase and 4.90 breaths/min for CapnoBase→BIDMC. Uncertainty–error Spearman correlation fell from 0.48 and 0.41 internally to 0.15 and 0.09 externally, and gating yielded only modest improvement. A fixed raw-spectral baseline outperformed the learned model in both transfer directions. Conclusions: Confidence measured within the development cohort did not establish safety after transfer. PPG-derived respiratory rate needs external calibration before it can guide rehabilitation feedback, even when point accuracy appears acceptable internally. Public benchmarks should report bidirectional transfer and risk–coverage behavior alongside random or subject-wise internal splits.

Preprints.org
Good health and well-being
Non-Invasive Vital Sign Monitoring
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon