Grammar vs Bits: Format Contracts and Weight Precision in Small-Model Tool Calling
Does strict grammar decoding recover what Q4 quantization loses in small-model tool calling? We find no clear quantization-specific recovery. On the primary JSON-contract point estimates, whole-response constraints helped F16 at least as much as Q4 for both Llama models; only Llama-1B's recovery interval is nominally negative, and Llama-3B's interval includes zero. In a separate sensitivity on the same 60 parallel cases, Q4 whole-strict accuracy was 1.0% (primary JSON) versus 62.3% (Pythonic) for Llama-1B and 0.0% versus 88.3% for Llama-3B. Paired gains of 61.3 [48.7, 73.3] and 88.3 [80.0, 95.7] points far exceed Pythonic F16-to-Q4 gaps of 6.0 and 3.3 points. A format contract means the prompt, parser and grammar. This nonrandomized sensitivity changes all three, separately from the primary BFCL conceptual replication; it does not isolate a parser-only causal effect. Quantization-specific recovery and its change from JSON remain inconclusive in the sensitivity. Intervals are pointwise, unadjusted and specific to this setup. No primary free-decoding precision-loss interval excludes zero. We release frozen inputs, responses, code and paired uncertainty estimates.
Authors
- Mohith M (ORCID: https://orcid.org/0009-0003-2978-3045)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-03
- DOI
- https://doi.org/10.5281/zenodo.23926881
- Primary Topic
- Natural Language Processing Techniques
- Type
- preprint