Ternary KV Cache for a 27B Model on a Phone, Preliminary Functional Report
Preliminary technical note on a ternary KV cache for a 27B parameter Qwen3.5-class model. The cache stores both K and V in TQ1_0 or TQ2_0 format and runs through the non-flash attention path with Q8_K activations. The model is Ternary Bonsai 2 27B with 64 blocks, 16 full attention layers, 48 SSM layers, and head_dim 256. Functional correctness was tested on a Windows PC and on one Android phone. Both cache types produced coherent text. On the phone the TQ2_0 cache reached 6.14 to 6.37 tok/s prompt processing and 3.19 to 3.27 tok/s generation on a 32 token prompt and 8 generated tokens. The TQ1_0 cache was about 7 percent slower than q8_0 on decode and saved memory. The phone reached 67 to 68 C in about 30 seconds. The note records what is confirmed and what is not. Sustained performance, a real 32k context run, the MIUI application path, other SoCs, and GPU support remain open. No priority claim is made. Contents include the note, the TQ1_0 and TQ2_0 2x2 ARM kernel patches, the llama.rn ternary KV patch, a SHA256 manifest, and an OpenTimestamps proof. Text license CC BY 4.0. Code license MIT. Public handle figkcoci. Version 0.1.1 replaces the initial pending OpenTimestamps proofs with upgraded proofs. The note, manifest and archive are unchanged from 0.1.0. The upgraded proofs contain BitcoinBlockHeaderAttestation(974483). The calendars committed the proofs to Bitcoin transactions 6793ee832ed4620edb3ec75abefe6400564398149817d882cbc63962cb6e4040 and f098822a94cde8d206f411d8286ea769c9638d95d4b142e214ea56865b3f7282 in block 974483, whose merkle root is bd0ad171564ea008cc9206e6fb0929afac7e9e7b6dda84b3a41d2cad0942fdac. The record also includes BITCOIN_PROOF.txt with the block hash, merkle root, transaction ids and verification commands. Version 0.1.2 corrects an indexing error in the TQ1_0 2x2 ARM kernel that was present in 0.1.1. The decoder used q[j*5+k] instead of q[k*2+j]. The error affected even-length prompt prefill. Single-token decode uses the 1-row path and was not affected. The corrected patch and ERRATUM_2026-11-04.md are included. The 17.9 percent decode gain is withdrawn. The 44.5 percent prefill gain was measured with the incorrect kernel and is pending re-measurement. TQ2_0 2x2 is unaffected. The erratum, manifest and archive proofs are in Bitcoin block 974522. The unchanged note keeps its 974483 proof.
Authors
- Egor Pobozhii
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-04
- DOI
- https://doi.org/10.5281/zenodo.23937993
- Primary Topic
- Advanced Neural Network Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00