CNCM: A Compact Causal-Hybrid Next-Concept Architecture for Edge-Oriented Generation, Code Infilling, and Structured Tool Calling
Designed and implemented CNCM, a compact dual-mode Transformer combining a zero-table Multiscale Residual Finite Scalar Quantizer (M-RFSQ), a factorized concept head, Continuous Scaled RoPE, and a shared Exact-AR head for edge inference. In resource-constrained benchmarks, M-RFSQ reduces quantizer parameters by 18.03× (99.04% vs. 4.96% single-seed reconstruction), 8-step Concept Mode achieves 2.26×–7.33× laptop ONNX/DirectML and 6.72×–8.85× GPU latency ratios over 64-step Exact-AR, and Fine-Grained MoE scaling reduces 4-domain BPE perplexity by 75.47 PPL. Includes a transparent artifact-based audit reporting short-budget AR controls and negative held-out task results on tiny checkpoints.
Authors
- Akib Adnan
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-21
- DOI
- https://doi.org/10.5281/zenodo.24419823
- Primary Topic
- Topic Modeling
- Type
- preprint