Beyond Transformers: A NumPy Benchmark of Language-Model Architectures Across 13 Paradigm Categories
NumPy benchmark reporting 171 broad-sweep configurations across 13 labeled paradigm categories, 17 deep-ablation runs, and 4 follow-up runs. In the mixed prose/code ablation, HyenaDilated-d64 records 2.8177 average BPC at 2.78 microseconds per token with 29,312 parameters, versus Transformer-d128 at 2.9998 average BPC and 7.39 microseconds per token with 163,584 parameters. ConfGated-SSM+PPM-d128 records 1.9998 BPC on prose. The accompanying repository contains source code, result JSON files, and reproducibility material.
Authors
- Masoud Azizi (ORCID: https://orcid.org/0000-0002-8864-0533)
Institutions
- University of Tabriz (IR)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-21
- DOI
- https://doi.org/10.5281/zenodo.24424396
- Primary Topic
- Topic Modeling
- Type
- preprint