Native Binary-Weight Training Outperforms Zero-Shot Sign Projection in MNIST Diffusion

Does severe failure after projecting a diffusion model to one-bit weights show that its binary representation is unusable? We test that implication, rather than propose a new quantizer, in a small MNIST residual U-Net. We compare floating-point training, zero-shot per-channel sign/scale projection of a trained model, binary-weight training from random initialization, floating-point warm-started quantization-aware training (QAT), and a binary-weight/binary-activation core. The primary FP32, native W1A32, and projected W1A32 comparison fixes the parameterization, data, training budget where applicable, and 1,000-step ancestral sampler, with three independent training seeds. Native W1A32 training obtains a domain-feature Fréchet distance of 46.33 ± 17.24, which is 28.4-fold lower than matched zero-shot projection (1314.20 ± 314.96), but remains substantially worse than FP32 (3.15 ± 0.38). Warm QAT also recovers much of the projection loss, while the W1A1 core retains a large distributional gap. An elementary feasible-set argument explains why one projected floating-point solution is not an impossibility certificate for the binary hypothesis class. The supported claim is deliberately local: on MNIST, optimizing binary forwards reaches a much better W1A32 solution than this zero-shot projection baseline. This does not establish full W1A1 success, floating-point parity, or transfer to natural images. This record is an unreviewed technical report. The accompanying source archive contains the LaTeX manuscript, bibliography, and manuscript figures. Code, checkpoints, evaluation artifacts, and the Diffuino demonstration are available from the related project resources.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-17
DOI
https://doi.org/10.5281/zenodo.23424653
Primary Topic
Advanced Neural Network Applications
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Native Binary-Weight Training Outperforms Zero-Shot Sign Projection in MNIST Diffusion

Ayushman Bhattacharya, Nihal Gazi, Anwesha Chakraborty
Zenodo (CERN European Organization for Nuclear Research)
Advanced Neural Network Applications
preprint

Native Binary-Weight Training Outperforms Zero-Shot Sign Projection in MNIST Diffusion

Ayushman Bhattacharya, Nihal Gazi, Anwesha Chakraborty
preprint en

Abstract

Does severe failure after projecting a diffusion model to one-bit weights show that its binary representation is unusable? We test that implication, rather than propose a new quantizer, in a small MNIST residual U-Net. We compare floating-point training, zero-shot per-channel sign/scale projection of a trained model, binary-weight training from random initialization, floating-point warm-started quantization-aware training (QAT), and a binary-weight/binary-activation core. The primary FP32, native W1A32, and projected W1A32 comparison fixes the parameterization, data, training budget where applicable, and 1,000-step ancestral sampler, with three independent training seeds. Native W1A32 training obtains a domain-feature Fréchet distance of 46.33 ± 17.24, which is 28.4-fold lower than matched zero-shot projection (1314.20 ± 314.96), but remains substantially worse than FP32 (3.15 ± 0.38). Warm QAT also recovers much of the projection loss, while the W1A1 core retains a large distributional gap. An elementary feasible-set argument explains why one projected floating-point solution is not an impossibility certificate for the binary hypothesis class. The supported claim is deliberately local: on MNIST, optimizing binary forwards reaches a much better W1A32 solution than this zero-shot projection baseline. This does not establish full W1A1 success, floating-point parity, or transfer to natural images. This record is an unreviewed technical report. The accompanying source archive contains the LaTeX manuscript, bibliography, and manuscript figures. Code, checkpoints, evaluation artifacts, and the Diffuino demonstration are available from the related project resources.

Zenodo (CERN European Organization for Nuclear Research)
JIS University (IN)
Advanced Neural Network Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.