What Costs Rows in Small EZKL Transformer Circuits, and What It Costs in Quantized Quality
Published accounts argue that nonlinear operations can dominate zero-knowledge (ZK) inference cost, while matrix products can be relatively cheap. We measure how EZKL 23.0.5 allocates rows and advice columns for small causal transformers and what small jointly trained and compiled models cost in quantized quality. We also formalize one operation-counting heuristic as an explicit predictor; this is our own stand-in, not a published model. We compile 108 configurations (width 64–128, 1–4 layers, sequence length 8–32, four activation variants) and measure proof-domain sizes without making proving-time comparisons. All report two inner columns, so rows multiplied by this count preserve row rankings. The operation-count stand-in ranks compiled row counts well (Spearman $\rho{=}0.93$) but no better than the product of the dimensions ($0.96$), is uncalibrated (measured/modelled rows $0.12$–$0.53$), and predicts held-out widths, depths and sequence lengths worse ($24$–$38\%$ error) than a power law ($6$–$18\%$) or a calibrated additive model ($8$–$11\%$). On 16 out-of-range configurations (width 256–512) the stand-in’s fitted prediction errs by $122\%$ against $10\%$ for the dimension product, and the activation overheads persist (GELU $1.45$–$1.54\times$, ReLU $1.08$–$1.17\times$ the linear variant’s rows). For sequences up to 32, the FFN nonlinearity dominates the row overhead for GELU and poly2 (at width 128, ReLU $+7\%$, poly2 $+20\%$, GELU $+42\%$), but softmax grows with sequence length (from $4\%$ to $24\%$ at width 64 as the length goes from 8 to 64). EZKL’s domain size also depends on an implementation detail, the causal-mask fill value, which moved it from 19 to 26 at constant rows. In a WikiText-2 study (3 seeds, validation-selected checkpoints), ReLU, poly2 and GELU are within $1\%$ in perplexity, and larger models help within the tested protocol. Compiled circuits reach width 512 (the grid width 128) with at most four layers, and the mask-fill sweep spans domains 19–26. The large-vocabulary quality models were not compiled; for three matched 512-symbol models using ReLU, GELU and polynomial- square activations, test PPL after fixed-point execution at scales 11–13 is within $0.12\%$ of each model’s full-precision reference on 960 targets. Code and raw records are released.
Authors
- Nissan Pow
Publication Details
- Published
- 2026-11-16
- DOI
- https://doi.org/10.68852/pf-261116.000001
- Primary Topic
- Cryptography and Data Security
- Type
- article
- Field-Weighted Citation Impact
- 0.00