Precision Transition Dynamics: Standardizing BF16 to U8 Containers in Sub-Block Quantization for Edge LLM Inference

Journal Description Title: Precision Transition Dynamics: Standardizing BF16 to U8 Containers in Sub-Block Quantization for Edge LLM Inference Author: Andri Johandri (Independent AI Research Team, POKDARTIK, Pemalang, Indonesia / WiTi4Bit Project) Main Topic: Post-training quantization (PTQ) and memory container standardization for efficient Large Language Models (LLMs) on resource-constrained edge hardware. Core Contributions & Highlights: Precision Transition Dynamics (PTD) Framework: The paper introduces a deterministic, bit-level sub-block quantization framework that operates without requiring external calibration data. PTD leverages canonical binary routing for exponents and structured mapping of mantissas via mirror pairs combined with a strict round-half-up mechanism to eliminate stochastic rounding noise. Hybrid U8 Safetensors Container Architecture: It proposes a 1,000,000-element threshold to safeguard architectural stability. Sensitive layers (such as layer normalization and key-value projections) are isolated in split-FP16 format, while massive weight tensors are compressed into compact QnM blocks (18 bytes per 32-element sub-block). Empirical Evaluation Results: Evaluated on the Gemma-3-1B-IT architecture, the Q4M implementation achieved a 70.8% size reduction (3.42x compression ratio), shrinking model weights from 2.00 GB to 584.8 MB. Furthermore, the model maintained high logit fidelity with an average cosine similarity of 0.9556 alongside robust text generation capabilities. Resources & Repositories: Quantized Model Repository: Hugging Face - gemma-3-1b-it-qnm-4bit Inference Runtime Toolchain: GitHub - runtime_chat