Surprise-Annealed Dual-Tail Loss for Robust Noisy Labels Learning

Overparameterized deep neural networks (DNNs) rely heavily on high-quality labels, while conventional cross entropy (CE) tends to overfit incorrect annotations under label noise, leading to degraded generalization performance. Robust loss functions alleviate this issue by reducing sensitivity to noisy labels. However, strict symmetric losses often exhibit limited fitting capability, while full normalization may alter the optimization properties of the original loss. To address these limitations, we propose surprise-annealed dual-tail loss (SADT), a bounded multiclass classification objective constructed from predictive surprise. Specifically, SADT retains the original predictive surprise in the active score construction when the predicted probability is at least the chance level, progressively compresses moderate surprise through an inverse hyperbolic sine branch, and bounds the extreme tail using an arctangent branch. By combining a normalized active score with a log balanced passive score, SADT achieves an exact constant class sum. Theoretically, we establish its boundedness and prove that it preserves clean risk minimizers under uniform symmetric label noise with a corruption rate below (C − 1)/C. For class conditional and instance-dependent noise, we derive conditional robustness guarantees and an excess risk bound. Experiments show that SADT improves accuracy over NCE + NNCE by 2.64% and 3.16% on CIFAR-10 with 80% symmetric noise and WebVision, respectively, while showing weaker performance on Clothing1M. These results demonstrate the effectiveness of SADT under several challenging noise conditions while revealing its limitations in learning from difficult clean samples.

Authors

Institutions

Publication Details

Journal
Entropy
Published
2026-11-19
DOI
https://doi.org/10.3390/e28111279
Primary Topic
Machine Learning and Data Classification
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Surprise-Annealed Dual-Tail Loss for Robust Noisy Labels Learning

Yanzhan Chen, Yongqi Yuan, Qian Zhang, Ge Jin
Entropy
Machine Learning and Data Classification
article

Surprise-Annealed Dual-Tail Loss for Robust Noisy Labels Learning

Yanzhan Chen, Yongqi Yuan, Qian Zhang, Ge Jin
article en

Abstract

Overparameterized deep neural networks (DNNs) rely heavily on high-quality labels, while conventional cross entropy (CE) tends to overfit incorrect annotations under label noise, leading to degraded generalization performance. Robust loss functions alleviate this issue by reducing sensitivity to noisy labels. However, strict symmetric losses often exhibit limited fitting capability, while full normalization may alter the optimization properties of the original loss. To address these limitations, we propose surprise-annealed dual-tail loss (SADT), a bounded multiclass classification objective constructed from predictive surprise. Specifically, SADT retains the original predictive surprise in the active score construction when the predicted probability is at least the chance level, progressively compresses moderate surprise through an inverse hyperbolic sine branch, and bounds the extreme tail using an arctangent branch. By combining a normalized active score with a log balanced passive score, SADT achieves an exact constant class sum. Theoretically, we establish its boundedness and prove that it preserves clean risk minimizers under uniform symmetric label noise with a corruption rate below (C − 1)/C. For class conditional and instance-dependent noise, we derive conditional robustness guarantees and an excess risk bound. Experiments show that SADT improves accuracy over NCE + NNCE by 2.64% and 3.16% on CIFAR-10 with 80% symmetric noise and WebVision, respectively, while showing weaker performance on Clothing1M. These results demonstrate the effectiveness of SADT under several challenging noise conditions while revealing its limitations in learning from difficult clean samples.

EntropyVol. 28(11)
Nanjing Forestry University (CN), Jiangsu Open University (CN)
Machine Learning and Data Classification
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon