How hard is my imbalanced classification task? - An explainable supervised approach

• First end-to-end explainable framework predicting dataset difficulty and causes. • GPU-optimized complexity meta-features for overlap, noise, nonlinearity. • Novel methods are proposed for scalable complexity metrics in multiclass settings. • EBMs & GAMformer act as meta-models providing accuracy and interpretability. • Tested on 613 binary & 91 multiclass imbalanced datasets. Despite extensive work on imbalanced classification, the reasons why certain datasets prove more challenging than others remain poorly understood, owing to the intricate interplay of class imbalance with data irregularities such as class overlap, the presence of small disjuncts or noise. We address this gap by introducing the first end-to-end explainable framework that forecasts a dataset’s classification difficulty while simultaneously identifying and quantifying the key data irregularities dictating such assessment. Our contributions are threefold: (1) a suite of GPU-accelerated complexity metrics tailored for imbalanced domains, which act as meta-feature in the proposed framework; (2) two novel, scalable methodologies for computing complexity measures in multiclass settings; and (3) two explainable profiling models, one based on Explainable Boosting Machines (EBMs) and another on GAMformer, a state-of-the-art additive in-context learning model. On a benchmark of approximately 600 real-world binary and 120 multiclass datasets, our EBMs and GAMformer achieved a R 2 of 0.921 and 0.864 for binary classification problems and 0.899 and 0.799 for multiclass settings, respectively, while producing transparent explanations that align with both theoretical expectations and t-SNE visualizations. Finally, we showcase real-world applicability by accurately identifying when SMOTE and cost-sensitive Random Forests, two widely adopted technique for addressing imbalanced domains, will improve classification outcomes across varying data profiles. The proposed framework is computationally efficient (GAMformer requires no training), highly accurate, and readily interpretable, offering a powerful tool for dataset profiling, benchmarking, method development, and meta-learning in imbalanced classification.