CIRA: Cascaded Implicit Recurrent Attention

CIRA provides a structured alternative to conventional transformer attention: semantic state formation precedes contextual state formation, and attention operates over dynamically generated equilibrium representations rather than independently parameterized key-value memories. The architecture offers an interpretable state-machine decomposition in which semantic and contextual dynamics, fixed-point representations, and token prediction can be examined separately. This provides a framework for structured implicit dynamics and entropy-regularized attention to improve the semantic coherence, efficiency, and auditability of language models.

Authors

Publication Details

Journal
HAL (Le Centre pour la Communication Scientifique Directe)
Published
2026-09-11
DOI
https://doi.org/10.13140/rg.2.2.16058.35521
Primary Topic
Explainable Artificial Intelligence (XAI)
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

CIRA: Cascaded Implicit Recurrent Attention

Gary Nan Tie
HAL (Le Centre pour la Communication Scientifique Directe)
Explainable Artificial Intelligence (XAI)
preprint

CIRA: Cascaded Implicit Recurrent Attention

Gary Nan Tie
preprint en

Abstract

CIRA provides a structured alternative to conventional transformer attention: semantic state formation precedes contextual state formation, and attention operates over dynamically generated equilibrium representations rather than independently parameterized key-value memories. The architecture offers an interpretable state-machine decomposition in which semantic and contextual dynamics, fixed-point representations, and token prediction can be examined separately. This provides a framework for structured implicit dynamics and entropy-regularized attention to improve the semantic coherence, efficiency, and auditability of language models.

HAL (Le Centre pour la Communication Scientifique Directe)
Explainable Artificial Intelligence (XAI)
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.