RELAX: Reliability enhancement by limiting adverse eXertions

Abstract Accurate and well-calibrated confidence estimates are essential for the safe use of artificial intelligence (AI) in clinical imaging workflows, where overconfident errors can directly affect diagnostic decisions and patient outcomes. Conventional training objectives such as cross-entropy often encourage uniformly high confidence across all samples, leading to substantial miscalibration, particularly on rare pathologies, heterogeneous patient populations, and inherently ambiguous cases. Recent work addresses this by modifying loss functions to improve calibration, yet these approaches typically penalize errors equally across samples, including intrinsically difficult or noisy examples, which can contribute to suboptimal calibration. To address this limitation, we introduce the Perpetual Burden Score (PBS), a measure of sample difficulty derived from prediction instability observed during training. Building on PBS, we propose , a burden-aware optimization framework that down-weights chronically unstable (hard) samples during training, thereby reducing their influence on confidence formation. Across four medical imaging datasets spanning ophthalmology, histopathology, neuroimaging, and chest radiography, demonstrates overall improvements in calibration, while maintaining comparable diagnostic accuracy. For example, across 5 different architectural configurations on the ophthalmology dataset, on average, reduces Expected Calibration Error (ECE) and Maximum Calibration Error (MCE) by 2.38% and 11.72%, respectively, while the reduction in F1-score is observed at 0.86%. These findings suggest that incorporating prediction stability into the training process can improve the reliability of confidence estimates without additional inference-time cost. As an architecture-agnostic and lightweight framework, provides a practical approach for enhancing the trustworthiness of clinical AI systems.