Deep Fibration Attention

In fibration attention, a lifting simultaneously updates a query and abductively finds a key to explain a value. This all depends on the key-value pairing (k,v). By determining other keys that are highly similar to k, to also explain v, we can enhance key-value semantics to achieve more meaningful attention. Moreover, we can recursively layer keys upon keys in a tree, to deepen the set of value explanations.

Authors

Publication Details

Journal
SPIRE - Sciences Po Institutional REpository
Published
2025-01-01
DOI
https://doi.org/10.13140/rg.2.2.17946.30402
Primary Topic
Advanced Numerical Analysis Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Deep Fibration Attention

Gary Nan Tie
SPIRE - Sciences Po Institutional REpository
Advanced Numerical Analysis Techniques
article

Deep Fibration Attention

Gary Nan Tie
article en

Abstract

In fibration attention, a lifting simultaneously updates a query and abductively finds a key to explain a value. This all depends on the key-value pairing (k,v). By determining other keys that are highly similar to k, to also explain v, we can enhance key-value semantics to achieve more meaningful attention. Moreover, we can recursively layer keys upon keys in a tree, to deepen the set of value explanations.

SPIRE - Sciences Po Institutional REpository
Openalex Percentile: Top 72%
Advanced Numerical Analysis Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Deep Fibration Attention — Gary Nan Tie · SPIRE - Sciences Po Institutional REpository (2025) | TGRS Research Map | TGRS