Entropy-driven adaptive knowledge distillation with reliable candidate fusion for imbalanced binary classification

Abstract Class imbalance can impair minority-class recognition and probability reliability, while conventional knowledge distillation often relies on fixed knowledge-transfer settings and a fixed prediction form. In this study, we propose entropy-driven adaptive knowledge distillation with reliable candidate fusion (EAKD-RCF) for imbalanced binary classification. EAKD-RCF consists of two main stages. First, adaptive distillation separates predictive uncertainty from teacher reliability: uncertainty determines the sample-specific distillation temperature, whereas training-fold-level reliability regulates the overall strength of knowledge transfer. Second, student, teacher, and fused predictions are treated as complementary candidates, and the final prediction form is selected using internal validation by jointly considering imbalanced classification performance and probability quality. The proposed method was evaluated on nine public datasets using repeated stratified five-fold cross-validation. EAKD-RCF achieved mean MCC, F1-score, and G-Mean values of 61.96%, 72.55%, and 79.07%, respectively, while maintaining competitive ECE and Brier scores of 7.97% and 10.30%. Stage-wise and ablation analyses further showed complementary contributions from adaptive distillation and candidate selection. These results demonstrate strong average classification performance across heterogeneous datasets while maintaining competitive probability reliability.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-13
DOI
https://doi.org/10.21203/rs.3.rs-11193122/v1
Primary Topic
Imbalanced Data Classification Techniques
Type
preprint

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Entropy-driven adaptive knowledge distillation with reliable candidate fusion for imbalanced binary classification

Ping Jiang, Jue Zhang, Jihang Li, Hanye Liu et al.
1 links
Research Square
Imbalanced Data Classification Techniques
preprint

Entropy-driven adaptive knowledge distillation with reliable candidate fusion for imbalanced binary classification

Ping Jiang, Jue Zhang, Jihang Li, Hanye Liu, Lele Su, Hui Zhang
preprint en

Abstract

Abstract Class imbalance can impair minority-class recognition and probability reliability, while conventional knowledge distillation often relies on fixed knowledge-transfer settings and a fixed prediction form. In this study, we propose entropy-driven adaptive knowledge distillation with reliable candidate fusion (EAKD-RCF) for imbalanced binary classification. EAKD-RCF consists of two main stages. First, adaptive distillation separates predictive uncertainty from teacher reliability: uncertainty determines the sample-specific distillation temperature, whereas training-fold-level reliability regulates the overall strength of knowledge transfer. Second, student, teacher, and fused predictions are treated as complementary candidates, and the final prediction form is selected using internal validation by jointly considering imbalanced classification performance and probability quality. The proposed method was evaluated on nine public datasets using repeated stratified five-fold cross-validation. EAKD-RCF achieved mean MCC, F1-score, and G-Mean values of 61.96%, 72.55%, and 79.07%, respectively, while maintaining competitive ECE and Brier scores of 7.97% and 10.30%. Stage-wise and ablation analyses further showed complementary contributions from adaptive distillation and candidate selection. These results demonstrate strong average classification performance across heterogeneous datasets while maintaining competitive probability reliability.

Research Square
Yulin University (CN)
National Natural Science Foundation of China, Yakult Bio-Science Foundation, Yulin University, Key Research and Development Projects of Shaanxi Province
Imbalanced Data Classification Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon