Conformal reliability assessment of PPG-based ICU arrhythmia classification under class imbalance and patient heterogeneity

OBJECTIVE: Deep learning models for photoplethysmography (PPG)-based arrhythmia detection in intensive care are often evaluated by average accuracy, while conformal prediction (CP) is used to add marginal coverage guarantees. We ask whether aggregate accuracy and marginal conformal coverage are sufficient evidence of clinical reliability under ICU class imbalance and patient heterogeneity.

Approach. Using the MIMIC-III-Ext-PPG benchmark with 826 critically ill patients and over 1.1 million 30-second PPG segments, we conduct a conformal reliability audit of PPG-only arrhythmia classification. We deliberately use a stress-case classifier with superficially reassuring aggregate metrics but weak rare-class behavior, and test whether accuracy and marginal CP conceal per-class and per-patient failures. We evaluate two predictors, three global conformal score constructions (LAC, APS, and SAPS), class-conditional calibration, an empirical RC3P-style class-aware sensitivity analysis, prevalence reweighting, and signal-quality-aware variants, with uncertainty quantified by patient-clustered resampling.

Main results. The stress-case classifier attains 65.0% accuracy but only 0.388 balanced accuracy, revealing severe rare-class failure despite superficially reassuring aggregate performance. Under global CP, marginal coverage reaches 90.0%, yet critical-rhythm coverage is only 7.5%, with 267 of 268 critical-rhythm segments not assigned to the correct top class. This collapse persists across predictors and global scores (APS: 96.0% overall but 23.1% critical; SAPS: 90.2% overall but 7.8% critical) and survives full-benchmark prevalence reweighting. Class-conditional calibration raises critical-rhythm coverage to 0.825, but the patient-clustered interval [0.578, 0.985] shows that target-level reliability remains uncertified for the rarest class. More than a third of patients fall below target even among those with many segments. Signal quality is not the dominant failure axis: low-quality segments are not under-covered, and quality-weighted calibration offers no advantage over simple baselines.

Significance. In ICU PPG arrhythmia monitoring, marginal coverage is not clinical reliability. Systems should be evaluated by per-class and per-patient coverage with patient-clustered uncertainty before claims of trustworthiness.