Addressing Data Scarcity and Class Imbalance Using Active Learning and Synthetic Data Generation: Experimentation on Clinical Datasets

The rapid advancement of Machine Learning (ML) technologies has revolutionized healthcare, particularly in the domains of disease prediction and diagnosis. Nevertheless, some data-related issues, such as annotated data scarcity, low-quality labels, and class imbalance in the dataset, can compromise the effectiveness of the learning process. This paper first gives an overview of Active Learning (AL), which is a strategy in machine learning that helps models to achieve better performance using fewer labeled examples. It then reviews various strategies for addressing dataset imbalance, in particular focusing on data-level rebalancing approaches. The paper provides a review of existing works on active learning in the presence of class imbalance and on the integration of active learning with synthetic oversampling techniques. The central core of the paper is the implementation of four different strategies for data selection from publicly available clinical datasets: Random, Random + SMOTE, Uncertainty, and Uncertainty + SMOTE. We calculated and plotted balanced accuracy and average precision by iterating over 20 different seed values, using cumulative batches of labeled data, for each combination of dataset and classifier. Moreover we also carried out a statistical test to assess the extent to which the difference between the Random and Uncertainty distributions, and that between the Uncertainty and Uncertainty + SMOTE distributions, was statistically significant. Experiments show that uncertainty-based strategy can improve predictive performance compared with random acquisition, particularly with Random Forest, although the observed benefits vary across datasets, classifiers, performance metrics, and annotation budgets. The additional contribution of SMOTE is also context dependent and does not yield statistically significant improvements over uncertainty-based acquisition alone. For the WBCD dataset, it can be seen that Uncertainty + SMOTE is comparable to, or in some cases below, the uncertainty-only strategy. Metrics such as precision, recall, specificity, F1-score, AUROC and AUPRC are evaluated and provided.

Authors

Institutions

Publication Details

Journal
AI
Published
2026-11-13
DOI
https://doi.org/10.3390/ai7110485
Primary Topic
Imbalanced Data Classification Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Addressing Data Scarcity and Class Imbalance Using Active Learning and Synthetic Data Generation: Experimentation on Clinical Datasets

Allan Tucker, Mario Cannataro, Lorella Bottino, Chiara Zucco
AI
Imbalanced Data Classification Techniques
article

Addressing Data Scarcity and Class Imbalance Using Active Learning and Synthetic Data Generation: Experimentation on Clinical Datasets

Allan Tucker, Mario Cannataro, Lorella Bottino, Chiara Zucco
article en

Abstract

The rapid advancement of Machine Learning (ML) technologies has revolutionized healthcare, particularly in the domains of disease prediction and diagnosis. Nevertheless, some data-related issues, such as annotated data scarcity, low-quality labels, and class imbalance in the dataset, can compromise the effectiveness of the learning process. This paper first gives an overview of Active Learning (AL), which is a strategy in machine learning that helps models to achieve better performance using fewer labeled examples. It then reviews various strategies for addressing dataset imbalance, in particular focusing on data-level rebalancing approaches. The paper provides a review of existing works on active learning in the presence of class imbalance and on the integration of active learning with synthetic oversampling techniques. The central core of the paper is the implementation of four different strategies for data selection from publicly available clinical datasets: Random, Random + SMOTE, Uncertainty, and Uncertainty + SMOTE. We calculated and plotted balanced accuracy and average precision by iterating over 20 different seed values, using cumulative batches of labeled data, for each combination of dataset and classifier. Moreover we also carried out a statistical test to assess the extent to which the difference between the Random and Uncertainty distributions, and that between the Uncertainty and Uncertainty + SMOTE distributions, was statistically significant. Experiments show that uncertainty-based strategy can improve predictive performance compared with random acquisition, particularly with Random Forest, although the observed benefits vary across datasets, classifiers, performance metrics, and annotation budgets. The additional contribution of SMOTE is also context dependent and does not yield statistically significant improvements over uncertainty-based acquisition alone. For the WBCD dataset, it can be seen that Uncertainty + SMOTE is comparable to, or in some cases below, the uncertainty-only strategy. Metrics such as precision, recall, specificity, F1-score, AUROC and AUPRC are evaluated and provided.

AIVol. 7(11)
Università degli Studi di Enna Kore (IT), Magna Graecia University (IT), Brunel University of London (GB)
Good health and well-being
Imbalanced Data Classification Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon