Validating Re-Clustering for Data Labeling with Human-in-the-Loop Feedback

This study presents a semi-supervised labeling framework integrating clustering algorithm and Human-in-the-Loop (HITL) methodology to improve labeling accuracy in highdimensional, noisy datasets. By leveraging an advanced density-based clustering algorithm, SSDBSCAN, and expert feedback, the framework enhanced data annotation across diverse datasets, including MIMIC III, Cancer Doc Classification, Hotel and Movie Reviews, Spam txt, and News Articles. Evaluations across HITL, non-HITL, and original labels confirm that the HITL framework achieved 98.25% accuracy on the MIMIC III dataset, significantly outperforming the non-HITL setup (96.25%). Similarly, on the Spam txt dataset, HITL attained an accuracy of 96.50%, compared to 93.70% without HITL and 89.63% using original labels. While datasets like Cancer and News posed challenges due to class imbalance and data complexity, HITL still demonstrated improved accuracy compared to non-HITL configurations, achieving 64.15% and 59.40% , respectively. These results highlight the framework’s validity, scalability, and reliability for semi-supervised labeling in large, unlabeled datasets.