Topology First: Sub-1-Bit LLM Compression by Separating Sign Topology from Metric

How far can a large language model be compressed below one bit per weight without heavy retraining? This paper presents NVG, a quantization method developed by the author and named after his Null-Vector Gravity framework. Instead of rounding every weight, NVG stores each layer as a pattern of ±1 signs plus a few 16-bit scales. It chooses the signs from the layer's own activations, solves the scales in closed form and then refines the signs, all without a gradient step. The storage layout comes from LittleBit (arXiv:2506.13771); NVG fills it differently. With no training, on Qwen3.5-0.8B at 1.0 bit per weight, WikiText-2 perplexity is 656 for NVG against 2,719,570 for LittleBit's data-free initializer in the same container. The 8-bit reference scores 24.27, so neither file is usable without training. After training on 10.5 M tokens against its own 8-bit file on one GPU, Qwen3.8-27B shrinks from 27.38 GB to 4.89 GB, with its layers at 0.55 bits per weight and 1.43 over the whole file. It decodes at 71-76 tok/s on one RTX PRO 6000, and its WikiText-2 perplexity is 29.90 against 8.48 for the 8-bit file. Short factual answers can be right, but longer ones repeat and contain errors, so it is not a working model yet. The paper gives the commands, the data and the limits of the method.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-03
DOI
https://doi.org/10.5281/zenodo.23925182
Primary Topic
Advanced Neural Network Applications
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Topology First: Sub-1-Bit LLM Compression by Separating Sign Topology from Metric

Oleg Yuryevich Kirichenko
Zenodo (CERN European Organization for Nuclear Research)
Advanced Neural Network Applications
preprint

Topology First: Sub-1-Bit LLM Compression by Separating Sign Topology from Metric

Oleg Yuryevich Kirichenko
preprint en

Abstract

How far can a large language model be compressed below one bit per weight without heavy retraining? This paper presents NVG, a quantization method developed by the author and named after his Null-Vector Gravity framework. Instead of rounding every weight, NVG stores each layer as a pattern of ±1 signs plus a few 16-bit scales. It chooses the signs from the layer's own activations, solves the scales in closed form and then refines the signs, all without a gradient step. The storage layout comes from LittleBit (arXiv:2506.13771); NVG fills it differently. With no training, on Qwen3.5-0.8B at 1.0 bit per weight, WikiText-2 perplexity is 656 for NVG against 2,719,570 for LittleBit's data-free initializer in the same container. The 8-bit reference scores 24.27, so neither file is usable without training. After training on 10.5 M tokens against its own 8-bit file on one GPU, Qwen3.8-27B shrinks from 27.38 GB to 4.89 GB, with its layers at 0.55 bits per weight and 1.43 over the whole file. It decodes at 71-76 tok/s on one RTX PRO 6000, and its WikiText-2 perplexity is 29.90 against 8.48 for the 8-bit file. Short factual answers can be right, but longer ones repeat and contain errors, so it is not a working model yet. The paper gives the commands, the data and the limits of the method.

Zenodo (CERN European Organization for Nuclear Research)
Advanced Neural Network Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
Topology First: Sub-1-Bit LLM Compression by Separating Sign Topology from Metric — Oleg Yuryevich Kirichenko · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS