Resampling, Stacking Diversity, and Decision Threshold Selection on Noisy Imbalanced Tabular Data
Classifying rare events in tabular datasets becomes difficult when severe class imbalance coincides with missing values, extreme outliers, and non-linear feature interactions. Under the standard 0.50 probability cutoff, classifiers often fall into the accuracy paradox: overall specificity stays high while minority cases go undetected. To examine how common remedies behave under these conditions, this study benchmarks a noisy synthetic tabular dataset (N = 3,000, 5% baseline minority weight, ~4% missing values, ~2% outliers, and ~1.5% borderline label ambiguity) strictly inside 5-fold Stratified Cross-Validation without data leakage. Across six resampling strategies (No Resampling, ROS, RUS, SMOTE, ADASYN, and SMOTE-Tomek), SMOTE-Tomek reaches the highest minority recall (0.8007) and edges out standard SMOTE on PR-AUC (0.3755 vs. 0.3748), whereas No Resampling retains the strongest PR-AUC (0.5620) and Brier Score (0.0565). Comparing single models, a tree-only stacking ensemble, and a heterogeneous stacking ensemble (Random Forest + XGBoost + Support Vector Machine with a Logistic Regression meta-learner) shows why cross-paradigm pairing helps: SVM (RBF) errors correlate far less with Random Forest (0.4934) and XGBoost (0.5661) than the two tree models correlate with each other (0.6735). That error diversity gives heterogeneous stacking the highest recall (0.8007) and well-calibrated probabilities (Brier Score = 0.0976). Finally, shifting the decision cutoff from 0.50 to Youden's J (0.0748) lifts recall from 0.5351 to 0.8007 (cutting missed detections from 126 down to 54) and raises Balanced Accuracy from 0.7203 to 0.7670.
Authors
- Muhammad Ketsar Ali Abi Wahid
Institutions
- Universitas Siber Asia (ID)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-22
- DOI
- https://doi.org/10.5281/zenodo.24423840
- Primary Topic
- Imbalanced Data Classification Techniques
- Type
- preprint