Analysis of Threshold Configuration Effects on Compression Ratio and Logit Fidelity in Q4M Quantization of Gemma-3-270M-IT

Article Overview Title: Analysis of Threshold Configuration Effects on Compression Ratio and Logit Fidelity in Q4M Quantization of Gemma-3-270M-IT Authors: Andri Johandri and Erni Sulistiowati (Independent AI Research Team, POKDARTIK, Indonesia — WiTi4Bit Project) Core Subject: An extended experimental investigation complementing previous publications on Precision Transition Dynamics (PTD), specifically evaluating how different Threshold settings influence tensor categorization, compression ratios, and numerical/logit fidelity during the Q4M 4-bit quantization of the Gemma-3-270M-IT model. Key Experimental Setup & Methodology Baseline Model: Gemma-3-270M-IT in BF16 representation, consisting of 236 tensors with a total file size of 536,223,056 bytes. Quantization Format: Q4M format featuring a sub-block size of 32 elements, with an 18-byte storage overhead per block (2-byte FP16 scale +6 bytes of packed 4-bit data), translating to an effective storage of 4.5 bits per quantized element before factoring in path routing. Configurations Tested: Two distinct internal configurations designated as Threshold A and Threshold B (exact numerical values are intentionally omitted for configuration confidentiality). Key Findings & Results 1. Tensor Categorization & Compression Performance Threshold A: Routed 55 tensors to the Q4M path and 181 tensors to the split-FP16 path. Achieved a file size of 193,342,400 bytes. Delivered a higher compression ratio of 2.7742x and a storage reduction of 63.95% (effective bits per element: 5.7674). Threshold B: Routed only 1 tensor to the Q4M path and 235 tensors to the split-FP16 path. Achieved a file size of 295,083,928 bytes. Yielded a lower compression ratio of 1.8175x and a storage reduction of 44.98% (effective bits per element: 8.8035). 2. Numerical & Logit Fidelity Evaluation The study evaluated fidelity across multiple metrics (Cosine Similarity, RMSE, MAE, KL Divergence, and Negative Log-Likelihood across Softmax and DR9 tracks): Threshold B consistently exhibited greater logit similarity and higher numerical alignment with the baseline model. It demonstrated higher Cosine Similarity values alongside lower RMSE, MAE, KL Divergence, and NLL errors compared to Threshold A. Threshold A prioritized aggressive space reduction at the cost of wider numerical divergence. Conclusion & Trade-offs Compression vs. Fidelity Trade-off: The experiments prove a clear trade-off: aggressive compression (Threshold A) maximizes storage savings but decreases numerical fidelity, whereas conservative compression (Threshold B) preserves higher logit fidelity at the expense of storage efficiency. Scope and Intent: The study does not propose a new quantization algorithm. Instead, it provides valuable empirical evidence regarding the behavior of the PTD Threshold mechanism to support researchers and developers working on model compression for resource-constrained edge computing environments. Source Model https://huggingface.co/WiTi4Bit/gemma-3-270m-it-qnm-4bit-2m Threshold Bhttps://huggingface.co/WiTi4Bit/gemma-3-270m-it-qnm-4bit Threshold A