Robust Prediction-Reliability Assessment in Biomedical Sensing and Classification Under Data Scarcity Using Conformal Prediction and Statistical Randomization

Reliable assessment of prediction reliability is important in biomedical sensing and measurement applications, where machine learning models increasingly transform imaging, physiological, chemical, and molecular measurements into clinically relevant predictions. Such applications often operate under limited labeled data because sensor acquisition, experimental measurement, and expert annotation can be costly, while biological and measurement variability can further challenge model development. Under these conditions, reliability scores derived from standard inductive conformal prediction (ICP) can be sensitive to the particular division of available data into proper-training and calibration subsets. To address this issue, we propose three randomized ICP variants: bootstrapping ICP (BICP), randomized ICP (RICP), and out-of-bag ICP (OOB-ICP), to improve the informativeness and repeated-sampling stability of conformal-derived prediction-reliability scores under data scarcity. We benchmarked these methods using a controlled simulation and three real-world biomedical sensing and measurement applications: pneumonia diagnosis from X-ray radiography, alternative herbal medicine classification using a 16-channel semiconductor electronic-nose gas-sensing system, and breast cancer subtyping from RNA-seq molecular measurements. Results show that the proposed methods produce aggregate credibility scores that are informative of prediction correctness and less sensitive to randomness in training-data sampling. Empirical prediction-set coverage analyses further characterize the behavior of the aggregated conformity scores relative to nominal error levels without assuming that they retain the exact finite-sample validity of conventional conformal p-values. Among the evaluated methods, OOB-ICP achieved strong overall classification performance and informative, stable prediction-reliability scores, supporting statistical randomization as a practical strategy for stabilizing conformal-derived reliability assessment under limited training data in biomedical sensing and measurement applications.