Can Coarse-to-Fine Text Generators Run on Ternary Weights? Post-Training Ternarization of Next-Concept Models Collapses; Quantization-Aware Fine-Tuning Recovers 83%

Next-Concept Modeling (NCM) generates text coarse-to-fine: a multiscale residual vector-quantized (VQ) tokenizer turns a sequence into a hierarchy of discrete codes, and a transformer predicts one level per forward pass. We ask whether such a model survives BitNet b1.58-style ternary weights ({−1, 0, +1}) with 8-bit activations. Using the official NCM code and Sudoku-hard checkpoint (86.15% of boards solved on an A100), we find that (i) post-training quantization (PTQ) to ternary weights collapses the model to 0% for every variant in our sweep, while INT8 is lossless and INT4 loses 4.4 points; (ii) the collapse is consistent with the magnitude of the weight perturbation alone: isotropic Gaussian noise of matched relative Frobenius error tracks the integer quantizers, the solve rate falls to 50% at a relative error of about 0.28, and even the best elementwise ternary rounding of Gaussian weights incurs ≈0.44, so ternary rounding that only minimizes weight error cannot succeed here; (iii) quantization-aware fine-tuning from the full-precision (FP) checkpoint, using ≈2.5% of the pretraining sample budget, recovers 71.15 ± 0.40% (3 seeds), i.e. 82.6% of FP; (iv) none of five interventions (distillation from the FP teacher, coarse-level loss reweighting, per-channel scales, 16-bit activations, keeping the most sensitive block in FP) raises this ceiling, and a per-puzzle decomposition shows that the remaining gap is not concentrated in the coarsest decision. INT8 activations cost no measurable accuracy. Code, logs and all reported numbers: https://github.com/drFurkanGuven/ternary-ncm (archived at https://doi.org/10.5281/zenodo.24322640).

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-17
DOI
https://doi.org/10.5281/zenodo.24322789
Primary Topic
Advanced Neural Network Applications
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Can Coarse-to-Fine Text Generators Run on Ternary Weights? Post-Training Ternarization of Next-Concept Models Collapses; Quantization-Aware Fine-Tuning Recovers 83%

Furkan GÜVEN
3 links
Zenodo (CERN European Organization for Nuclear Research)
Advanced Neural Network Applications
preprint

Can Coarse-to-Fine Text Generators Run on Ternary Weights? Post-Training Ternarization of Next-Concept Models Collapses; Quantization-Aware Fine-Tuning Recovers 83%

Furkan GÜVEN
preprint en

Abstract

Next-Concept Modeling (NCM) generates text coarse-to-fine: a multiscale residual vector-quantized (VQ) tokenizer turns a sequence into a hierarchy of discrete codes, and a transformer predicts one level per forward pass. We ask whether such a model survives BitNet b1.58-style ternary weights ({−1, 0, +1}) with 8-bit activations. Using the official NCM code and Sudoku-hard checkpoint (86.15% of boards solved on an A100), we find that (i) post-training quantization (PTQ) to ternary weights collapses the model to 0% for every variant in our sweep, while INT8 is lossless and INT4 loses 4.4 points; (ii) the collapse is consistent with the magnitude of the weight perturbation alone: isotropic Gaussian noise of matched relative Frobenius error tracks the integer quantizers, the solve rate falls to 50% at a relative error of about 0.28, and even the best elementwise ternary rounding of Gaussian weights incurs ≈0.44, so ternary rounding that only minimizes weight error cannot succeed here; (iii) quantization-aware fine-tuning from the full-precision (FP) checkpoint, using ≈2.5% of the pretraining sample budget, recovers 71.15 ± 0.40% (3 seeds), i.e. 82.6% of FP; (iv) none of five interventions (distillation from the FP teacher, coarse-level loss reweighting, per-channel scales, 16-bit activations, keeping the most sensitive block in FP) raises this ceiling, and a per-puzzle decomposition shows that the remaining gap is not concentrated in the coarsest decision. INT8 activations cost no measurable accuracy. Code, logs and all reported numbers: https://github.com/drFurkanGuven/ternary-ncm (archived at https://doi.org/10.5281/zenodo.24322640).

Zenodo (CERN European Organization for Nuclear Research)
Fırat University (TR)
Advanced Neural Network Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon