What Costs Rows in Small EZKL Transformer Circuits, and What It Costs in Quantized Quality

Published accounts argue that nonlinear operations can dominate zero-knowledge (ZK) inference cost, while matrix products can be relatively cheap. We measure how EZKL 23.0.5 allocates rows and advice columns for small causal transformers and what small jointly trained and compiled models cost in quantized quality. We also formalize one operation-counting heuristic as an explicit predictor; this is our own stand-in, not a published model. We compile 108 configurations (width 64–128, 1–4 layers, sequence length 8–32, four activation variants) and measure proof-domain sizes without making proving-time comparisons. All report two inner columns, so rows multiplied by this count preserve row rankings. The operation-count stand-in ranks compiled row counts well (Spearman $\rho{=}0.93$) but no better than the product of the dimensions ($0.96$), is uncalibrated (measured/modelled rows $0.12$–$0.53$), and predicts held-out widths, depths and sequence lengths worse ($24$–$38\%$ error) than a power law ($6$–$18\%$) or a calibrated additive model ($8$–$11\%$). On 16 out-of-range configurations (width 256–512) the stand-in’s fitted prediction errs by $122\%$ against $10\%$ for the dimension product, and the activation overheads persist (GELU $1.45$–$1.54\times$, ReLU $1.08$–$1.17\times$ the linear variant’s rows). For sequences up to 32, the FFN nonlinearity dominates the row overhead for GELU and poly2 (at width 128, ReLU $+7\%$, poly2 $+20\%$, GELU $+42\%$), but softmax grows with sequence length (from $4\%$ to $24\%$ at width 64 as the length goes from 8 to 64). EZKL’s domain size also depends on an implementation detail, the causal-mask fill value, which moved it from 19 to 26 at constant rows. In a WikiText-2 study (3 seeds, validation-selected checkpoints), ReLU, poly2 and GELU are within $1\%$ in perplexity, and larger models help within the tested protocol. Compiled circuits reach width 512 (the grid width 128) with at most four layers, and the mask-fill sweep spans domains 19–26. The large-vocabulary quality models were not compiled; for three matched 512-symbol models using ReLU, GELU and polynomial- square activations, test PPL after fixed-point execution at scales 11–13 is within $0.12\%$ of each model’s full-precision reference on 960 targets. Code and raw records are released.

Authors

Publication Details

Published
2026-11-16
DOI
https://doi.org/10.68852/pf-261116.000001
Primary Topic
Cryptography and Data Security
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

What Costs Rows in Small EZKL Transformer Circuits, and What It Costs in Quantized Quality

Nissan Pow
3 links
Cryptography and Data Security
article

What Costs Rows in Small EZKL Transformer Circuits, and What It Costs in Quantized Quality

Nissan Pow
article en

Abstract

Published accounts argue that nonlinear operations can dominate zero-knowledge (ZK) inference cost, while matrix products can be relatively cheap. We measure how EZKL 23.0.5 allocates rows and advice columns for small causal transformers and what small jointly trained and compiled models cost in quantized quality. We also formalize one operation-counting heuristic as an explicit predictor; this is our own stand-in, not a published model. We compile 108 configurations (width 64–128, 1–4 layers, sequence length 8–32, four activation variants) and measure proof-domain sizes without making proving-time comparisons. All report two inner columns, so rows multiplied by this count preserve row rankings. The operation-count stand-in ranks compiled row counts well (Spearman $\rho{=}0.93$) but no better than the product of the dimensions ($0.96$), is uncalibrated (measured/modelled rows $0.12$–$0.53$), and predicts held-out widths, depths and sequence lengths worse ($24$–$38\%$ error) than a power law ($6$–$18\%$) or a calibrated additive model ($8$–$11\%$). On 16 out-of-range configurations (width 256–512) the stand-in’s fitted prediction errs by $122\%$ against $10\%$ for the dimension product, and the activation overheads persist (GELU $1.45$–$1.54\times$, ReLU $1.08$–$1.17\times$ the linear variant’s rows). For sequences up to 32, the FFN nonlinearity dominates the row overhead for GELU and poly2 (at width 128, ReLU $+7\%$, poly2 $+20\%$, GELU $+42\%$), but softmax grows with sequence length (from $4\%$ to $24\%$ at width 64 as the length goes from 8 to 64). EZKL’s domain size also depends on an implementation detail, the causal-mask fill value, which moved it from 19 to 26 at constant rows. In a WikiText-2 study (3 seeds, validation-selected checkpoints), ReLU, poly2 and GELU are within $1\%$ in perplexity, and larger models help within the tested protocol. Compiled circuits reach width 512 (the grid width 128) with at most four layers, and the mask-fill sweep spans domains 19–26. The large-vocabulary quality models were not compiled; for three matched 512-symbol models using ReLU, GELU and polynomial- square activations, test PPL after fixed-point execution at scales 11–13 is within $0.12\%$ of each model’s full-precision reference on 960 targets. Code and raw records are released.

Vol. 1
Cryptography and Data Security
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon