Caliper: a local inference engine with an exact mode
Caliper is a local inference engine that runs one dense model on one consumer card, by Aetherion Substrata Ltd. A resident 6-bit form holds a 14B dense model in about 12.9 GB so nothing streams during generation: measured 12.4 tokens per second on a 16 GB card, greedy, single stream, with no measurable quality loss against the bfloat16 reference (perplexity 51.90 vs 51.93; 26 of 40 on a checkable quiz, both). An exact mode streams the original bfloat16 weights from system RAM to verify each run, so every token is bit-identical to the full-precision reference. Code and release: https://github.com/Aetherion-Substrata/Caliper KIZ - SOLO DEV - Aetherion Substrata Ltd - FIZZNETWORK
Authors
- KIZ
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-05
- DOI
- https://doi.org/10.5281/zenodo.23990337
- Primary Topic
- Advanced Neural Network Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00