A proof of concept for enhancing branchynet with trainable reference-vector-based early exits

Abstract As machine learning research advances, models continue to grow in both quality and scale. This trend increases computational costs and complicates large-scale deployment. Although various techniques reduce model size or accelerate inference with minimal accuracy loss, they are often complex to use and require substantial structural modifications. We present RefNet, a lightweight early exit framework that replaces BranchyNet’s computationally heavier auxiliary classifier heads with learnable, class-wise reference vectors. At each exit, RefNet computes the Euclidean distance between an intermediate feature representation and a small set of trainable per-class reference vectors and uses the resulting similarity scores both for classification and for the early-exit decision. In a proof-of-concept study using a ResNet-50 backbone on CIFAR-10, CIFAR-100, SVHN, and STL-10, with results averaged over five independent training and evaluation runs for each configuration, RefNet achieves inference speedups comparable to or greater than those of BranchyNet, while incurring lower training and parameter overhead: on average, it increases the parameter count by only 0.2% (vs. 1.7% for BranchyNet) and the training time by 1.1% (vs. 2.3% for BranchyNet). At its default operating point, using two early exits with confidence thresholds of 70% and 80%, respectively, RefNet achieves an inference speedup of up to 1.67 $$\times $$ on SVHN, with a mean accuracy reduction of only 0.14 percentage points relative to the vanilla ResNet-50. More aggressive thresholds increase the speedup on SVHN to 1.70 $$\times $$ at a correspondingly larger but still moderate accuracy cost. Welch’s t -tests over the five runs show that RefNet’s reduction in inference time relative to BranchyNet is statistically significant on all four datasets ( $$p\le 0.001$$ ), whereas the SVHN accuracy difference to the vanilla ResNet-50 at the default operating point is not ( $$p=0.25$$ ). These results demonstrate the feasibility of similarity-based early exit decision making as a lightweight alternative to learned classifier heads. RefNet does not cache or reuse computation across inference requests.

Authors

Publication Details

Journal
Discover Artificial Intelligence
Published
2026-11-17
DOI
https://doi.org/10.1007/s44163-026-02676-0
Primary Topic
Advanced Neural Network Applications
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

A proof of concept for enhancing branchynet with trainable reference-vector-based early exits

Vanessa Borst, Samuel Kounev, L.-C. Horn, Simon Engel et al.
Discover Artificial Intelligence
Advanced Neural Network Applications
article

A proof of concept for enhancing branchynet with trainable reference-vector-based early exits

Vanessa Borst, Samuel Kounev, L.-C. Horn, Simon Engel, Thomas Prantl, Phillip Ginzel
article en

Abstract

Abstract As machine learning research advances, models continue to grow in both quality and scale. This trend increases computational costs and complicates large-scale deployment. Although various techniques reduce model size or accelerate inference with minimal accuracy loss, they are often complex to use and require substantial structural modifications. We present RefNet, a lightweight early exit framework that replaces BranchyNet’s computationally heavier auxiliary classifier heads with learnable, class-wise reference vectors. At each exit, RefNet computes the Euclidean distance between an intermediate feature representation and a small set of trainable per-class reference vectors and uses the resulting similarity scores both for classification and for the early-exit decision. In a proof-of-concept study using a ResNet-50 backbone on CIFAR-10, CIFAR-100, SVHN, and STL-10, with results averaged over five independent training and evaluation runs for each configuration, RefNet achieves inference speedups comparable to or greater than those of BranchyNet, while incurring lower training and parameter overhead: on average, it increases the parameter count by only 0.2% (vs. 1.7% for BranchyNet) and the training time by 1.1% (vs. 2.3% for BranchyNet). At its default operating point, using two early exits with confidence thresholds of 70% and 80%, respectively, RefNet achieves an inference speedup of up to 1.67 $$\times $$ on SVHN, with a mean accuracy reduction of only 0.14 percentage points relative to the vanilla ResNet-50. More aggressive thresholds increase the speedup on SVHN to 1.70 $$\times $$ at a correspondingly larger but still moderate accuracy cost. Welch’s t -tests over the five runs show that RefNet’s reduction in inference time relative to BranchyNet is statistically significant on all four datasets ( $$p\le 0.001$$ ), whereas the SVHN accuracy difference to the vanilla ResNet-50 at the default operating point is not ( $$p=0.25$$ ). These results demonstrate the feasibility of similarity-based early exit decision making as a lightweight alternative to learned classifier heads. RefNet does not cache or reuse computation across inference requests.

Discover Artificial IntelligenceVol. 6(1)
Julius-Maximilians-Universität Würzburg
Advanced Neural Network Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon