A statistical matching approach to handling imbalanced datasets using binary classifiers

Class imbalance remains a major challenge for classification algorithms because conventional models tend to favour the majority class. Although statistical matching has been widely applied in data integration, its potential for addressing class imbalance has received limited attention. This study proposes StatMatch, a novel undersampling approach that adapts statistical matching to identify representative majority-class observations by pairing existing minority and majority observations on the basis of an exact matching criterion that evaluates the degree of agreement or disagreement among the chosen matching variables to generate balanced datasets without generating synthetic observations. StatMatch was evaluated against the Synthetic Minority Oversampling Technique (SMOTE) via Logistic Regression and Artificial Neural Network classifiers on four real-world imbalanced datasets. Performance was assessed using F-measure and Area Under the Receiver Operating Characteristic Curve (AUC) based on repeated stratified 10-fold cross-validation. Statistical differences between the two approaches were examined using paired Wilcoxon signed-rank exact tests. SMOTE generally achieved higher performance than StatMatch. For the Blood dataset, SMOTE obtained higher F-measure values for Logistic Regression (68.7 ± 3.7%) and ANN (68.8 ± 2.0%) compared with StatMatch (54.2 ± 5.6% and 57.4 ± 4.9%, respectively), with significant differences ( p ≤ 0.004). Similarly, SMOTE achieved higher AUC values (76.3 ± 3.1% and 75.8 ± 3.1%) than StatMatch (57.6 ± 5.1% and 56.5 ± 5.3%) for both classifiers ( p = 0.002). On the Breast dataset, both methods achieved high AUC performance using Logistic Regression (SMOTE: 99.5 ± 0.3%; StatMatch: 99.2 ± 0.3%), with no significant difference ( p = 0.065). For the Parkinson dataset, StatMatch slightly outperformed SMOTE with ANN in terms of F-measure (76.2 ± 15.6% vs. 75.0 ± 9.1%) and AUC (84.4 ± 12.7% vs. 82.9 ± 9.2%), although the differences were not statistically significant ( p > 0.05). The findings demonstrate that SMOTE remains a strong baseline method for improving classification performance on imbalanced datasets. However, StatMatch represents a promising complementary strategy for imbalanced binary classification, particularly in applications where maintaining data authenticity is desirable. Future work should investigate its performance on larger and more diverse datasets, additional classifiers, and multiclass classification problems.