CIRA: Cascaded Implicit Recurrent Attention
CIRA provides a structured alternative to conventional transformer attention: semantic state formation precedes contextual state formation, and attention operates over dynamically generated equilibrium representations rather than independently parameterized key-value memories. The architecture offers an interpretable state-machine decomposition in which semantic and contextual dynamics, fixed-point representations, and token prediction can be examined separately. This provides a framework for structured implicit dynamics and entropy-regularized attention to improve the semantic coherence, efficiency, and auditability of language models.
Authors
- Gary Nan Tie
Publication Details
- Journal
- HAL (Le Centre pour la Communication Scientifique Directe)
- Published
- 2026-09-11
- DOI
- https://doi.org/10.13140/rg.2.2.16058.35521
- Primary Topic
- Explainable Artificial Intelligence (XAI)
- Type
- preprint