Sampling Imbalanced Data with Bilevel Optimization

Imbalanced datasets, where minority classes are underrepresented compared to majority classes, present significant challenges in machine learning, often leading to biased models and poor classification of critical minority data. This dissertation introduces two novel sampling methods grounded in bilevel optimization to address these issues. The first method, Majority Undersampling with Bilevel Optimization (MUBO), improves classification without relying on any synthetic minority data by strategically undersampling majority data to enhance feature diversity and reduce class overlap. While effective on larger datasets, MUBO's performance declines with smaller datasets due to too few minority datapoints. To address these limitations, the second method, Multi-Objective Optimization for Data Sampling (MOODS), integrates a small amount of carefully selected synthetic data generated via SVM-SMOTE. This approach ensures that synthetic samples increase minority class diversity while also using bilevel optimization to avoid harmful overlap. A key contribution of this work is a novel metric designed to quantify the balance, diversity, and overlap of training datasets - critical properties for robust model performance. This `$\gamma / \epsilon / \delta$ balanced diversity metric' provides a framework for evaluating and comparing sampling methods, moving beyond traditional performance metrics like $F1$ scores to offer deeper insights into the quality of training data. Experimental results across multiple benchmark datasets demonstrate that both methods, alongside the new evaluation metric, outperform existing techniques, offering robust, generalizable solutions for imbalanced data classification.