Deep Fibration Attention
In fibration attention, a lifting simultaneously updates a query and abductively finds a key to explain a value. This all depends on the key-value pairing (k,v). By determining other keys that are highly similar to k, to also explain v, we can enhance key-value semantics to achieve more meaningful attention. Moreover, we can recursively layer keys upon keys in a tree, to deepen the set of value explanations.
Authors
- Gary Nan Tie
Publication Details
- Journal
- SPIRE - Sciences Po Institutional REpository
- Published
- 2025-01-01
- DOI
- https://doi.org/10.13140/rg.2.2.17946.30402
- Primary Topic
- Advanced Numerical Analysis Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00