ADAPTIVE INFERENCE THRESHOLDS AND SELF-DISTILLATION SHAPE CLASSIFICATION ACCURACY ACROSS COMPUTATIONALLY CONSTRAINED LANGUAGE MODEL BENCHMARKS
This analysis examines how adaptive computation configurations relate to classification accuracy in language models, informing international debates on computational efficiency. A descriptive reanalysis uses 20 published configuration results from Chinese Book review and English Yelp.P benchmarks reported in 2020, comparing routing arrangements, exit thresholds, and matched distillation settings. Adaptive routing exceeds selected fixed depth alternatives by 0.56 to 2.81 percentage points, with fewer operations in all three computationally usable comparisons. Raising exit thresholds reduces accuracy across both benchmarks, while endpoint computational reductions range from 62.77 to 77.87 percent among usable comparisons. Self-distillation improves accuracy in all four matched pairs, averaging 0.478 percentage points, with larger advantages for Book review. Accuracy losses concentrate differently across threshold intervals, indicating benchmark dependent stopping costs. These descriptive patterns distinguish selective processing, prediction acceptance, and intermediate supervision as complementary configuration decisions. The contribution refines the resource allocation perspective through transparent comparisons of existing evidence. Deployment teams should evaluate threshold intervals separately and compare accuracy alongside verified computational requirements within each intended task.
Authors
- Divya Vishwanath & Paul Johnson Asamoah
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-08
- DOI
- https://doi.org/10.5281/zenodo.24049706
- Primary Topic
- Advanced Neural Network Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00