Tron-1B: Fast, Calibrated Typed Decisions with a Set-Attention Option Head

Applications built around large language models make many small decisions: which queue a ticket belongs to, whether a message is a prompt-injection attempt, how urgent a request is, whether an agent's step succeeded. Routing each of these through a generative LLM is slow, costly and hard to calibrate. We present Tron-1B, a 1.06-billion-parameter bidirectional encoder with a purpose-built decision head that answers typed questions (choose one option, rate on an ordered scale, or answer yes/no) about arbitrary text or JSON in a single forward pass, returning a calibrated probability for every option. The head pools each option's span, lets options attend to each other and to the question through a small permutation-equivariant set-attention option mixer, and scores them with a pairwise MLP; temperatures are fitted per question type and option count. A companion runtime, troncore, adds cross-request batching, an elimination tournament for more than 128 options, sliding windows for long inputs, and confidence-based abstention. Trained for 13 hours on one NVIDIA DGX Spark on 1.19M typed questions drawn from 368 public datasets, Tron-1B reaches 94.0% on Banking77, 93.9% on AG News, 92.9% on DAIR Emotion and 79.6% on the typed-decisions benchmark, against third-party measurements of 87.0%, 91.0%, 48.0% and 72.7% for the commercial TypeSafe Jev 1.13.0 model, with a median latency of 16.7 ms per decision. This compares a model trained on those benchmarks' training splits with a zero-shot service. We report a data-contamination audit of every test set: overlap is below 1% for the four headline benchmarks but reaches 31.8% for MASSIVE, which we exclude from held-out claims. We also describe a gated nightly distillation loop that improves the model from its own traffic. Model weights (CC BY-NC 4.0) and the troncore runtime (Apache-2.0): https://huggingface.co/SamCodeManMk2/tron-1b

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-09-30
DOI
https://doi.org/10.5281/zenodo.23066522
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
preprint

Tron-1B: Fast, Calibrated Typed Decisions with a Set-Attention Option Head

Samir Sengupta
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

Tron-1B: Fast, Calibrated Typed Decisions with a Set-Attention Option Head

Samir Sengupta
preprint en

Abstract

Applications built around large language models make many small decisions: which queue a ticket belongs to, whether a message is a prompt-injection attempt, how urgent a request is, whether an agent's step succeeded. Routing each of these through a generative LLM is slow, costly and hard to calibrate. We present Tron-1B, a 1.06-billion-parameter bidirectional encoder with a purpose-built decision head that answers typed questions (choose one option, rate on an ordered scale, or answer yes/no) about arbitrary text or JSON in a single forward pass, returning a calibrated probability for every option. The head pools each option's span, lets options attend to each other and to the question through a small permutation-equivariant set-attention option mixer, and scores them with a pairwise MLP; temperatures are fitted per question type and option count. A companion runtime, troncore, adds cross-request batching, an elimination tournament for more than 128 options, sliding windows for long inputs, and confidence-based abstention. Trained for 13 hours on one NVIDIA DGX Spark on 1.19M typed questions drawn from 368 public datasets, Tron-1B reaches 94.0% on Banking77, 93.9% on AG News, 92.9% on DAIR Emotion and 79.6% on the typed-decisions benchmark, against third-party measurements of 87.0%, 91.0%, 48.0% and 72.7% for the commercial TypeSafe Jev 1.13.0 model, with a median latency of 16.7 ms per decision. This compares a model trained on those benchmarks' training splits with a zero-shot service. We report a data-contamination audit of every test set: overlap is below 1% for the four headline benchmarks but reaches 31.8% for MASSIVE, which we exclude from held-out claims. We also describe a gated nightly distillation loop that improves the model from its own traffic. Model weights (CC BY-NC 4.0) and the troncore runtime (Apache-2.0): https://huggingface.co/SamCodeManMk2/tron-1b

Zenodo (CERN European Organization for Nuclear Research)
Peace, Justice and strong institutions
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Tron-1B: Fast, Calibrated Typed Decisions with a Set-Attention Option Head — Samir Sengupta · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS