Grammar vs Bits: Format Contracts and Weight Precision in Small-Model Tool Calling

Does strict grammar decoding recover what Q4 quantization loses in small-model tool calling? We find no clear quantization-specific recovery. On the primary JSON-contract point estimates, whole-response constraints helped F16 at least as much as Q4 for both Llama models; only Llama-1B's recovery interval is nominally negative, and Llama-3B's interval includes zero. In a separate sensitivity on the same 60 parallel cases, Q4 whole-strict accuracy was 1.0% (primary JSON) versus 62.3% (Pythonic) for Llama-1B and 0.0% versus 88.3% for Llama-3B. Paired gains of 61.3 [48.7, 73.3] and 88.3 [80.0, 95.7] points far exceed Pythonic F16-to-Q4 gaps of 6.0 and 3.3 points. A format contract means the prompt, parser and grammar. This nonrandomized sensitivity changes all three, separately from the primary BFCL conceptual replication; it does not isolate a parser-only causal effect. Quantization-specific recovery and its change from JSON remain inconclusive in the sensitivity. Intervals are pointwise, unadjusted and specific to this setup. No primary free-decoding precision-loss interval excludes zero. We release frozen inputs, responses, code and paired uncertainty estimates.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-03
DOI
https://doi.org/10.5281/zenodo.23926881
Primary Topic
Natural Language Processing Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Grammar vs Bits: Format Contracts and Weight Precision in Small-Model Tool Calling

Mohith M
Zenodo (CERN European Organization for Nuclear Research)
Natural Language Processing Techniques
preprint

Grammar vs Bits: Format Contracts and Weight Precision in Small-Model Tool Calling

Mohith M
preprint en

Abstract

Does strict grammar decoding recover what Q4 quantization loses in small-model tool calling? We find no clear quantization-specific recovery. On the primary JSON-contract point estimates, whole-response constraints helped F16 at least as much as Q4 for both Llama models; only Llama-1B's recovery interval is nominally negative, and Llama-3B's interval includes zero. In a separate sensitivity on the same 60 parallel cases, Q4 whole-strict accuracy was 1.0% (primary JSON) versus 62.3% (Pythonic) for Llama-1B and 0.0% versus 88.3% for Llama-3B. Paired gains of 61.3 [48.7, 73.3] and 88.3 [80.0, 95.7] points far exceed Pythonic F16-to-Q4 gaps of 6.0 and 3.3 points. A format contract means the prompt, parser and grammar. This nonrandomized sensitivity changes all three, separately from the primary BFCL conceptual replication; it does not isolate a parser-only causal effect. Quantization-specific recovery and its change from JSON remain inconclusive in the sensitivity. Intervals are pointwise, unadjusted and specific to this setup. No primary free-decoding precision-loss interval excludes zero. We release frozen inputs, responses, code and paired uncertainty estimates.

Zenodo (CERN European Organization for Nuclear Research)
Natural Language Processing Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
Grammar vs Bits: Format Contracts and Weight Precision in Small-Model Tool Calling — Mohith M · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS