STRATA-HeadQuotient: Functional Localization of One Quarter of Global KV Heads with Typed Predicate-Graph Computation at 8192-Token Context

Global attention combines long-range lexical access with semantic-relation retrieval, making whole-layer replacement lossy. We present Strata-HeadQuotient, which statically assigns each key–value (KV) head to full-history attention (GLOBAL), 1024-token attention (LOCAL), or local attention plus an event-scoped typed graph read (LOCAL_GRAPH). For a 24-layer, 16-head decoder, we audited 384 heads and all 6903 pairs among 118 candidates before constrained subset selection. The Q25 export localized 96/384 heads: 81 were LOCAL and 15 were LOCAL_GRAPH. On 470 document-disjoint 8192-token windows (3.85 million tokens), graph-disabled Q25 perplexity was 1.00146× dense (95% document-bootstrap interval: 1.00131–1.00161); the 4096–8191-token ratio was 1.00201. On an untouched 2000-case semantic test, typed execution was 1998/2000, versus 374/2000 for a capacity-matched untyped graph. Two fresh adaptation/selection runs reproduced both directions, although every matched-capacity marginal graph-effect interval included zero. Q25 retained 78.125% of token-KV bytes and 98.44% of dense full-sequence throughput; a tested Q30 export failed. Thus, interaction-aware localization preserved 8192-token language-model quality, while a separate typed path controlled event-role behavior. The experiment does not establish that graph computation caused perplexity preservation.

Authors

Institutions

Publication Details

Journal
Machine Learning and Knowledge Extraction
Published
2026-09-04
DOI
https://doi.org/10.3390/make8090269
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

STRATA-HeadQuotient: Functional Localization of One Quarter of Global KV Heads with Typed Predicate-Graph Computation at 8192-Token Context

Мадина Мансурова, Nurgali Kadyrbek
Machine Learning and Knowledge Extraction
Topic Modeling
article

STRATA-HeadQuotient: Functional Localization of One Quarter of Global KV Heads with Typed Predicate-Graph Computation at 8192-Token Context

Мадина Мансурова, Nurgali Kadyrbek
article en

Abstract

Global attention combines long-range lexical access with semantic-relation retrieval, making whole-layer replacement lossy. We present Strata-HeadQuotient, which statically assigns each key–value (KV) head to full-history attention (GLOBAL), 1024-token attention (LOCAL), or local attention plus an event-scoped typed graph read (LOCAL_GRAPH). For a 24-layer, 16-head decoder, we audited 384 heads and all 6903 pairs among 118 candidates before constrained subset selection. The Q25 export localized 96/384 heads: 81 were LOCAL and 15 were LOCAL_GRAPH. On 470 document-disjoint 8192-token windows (3.85 million tokens), graph-disabled Q25 perplexity was 1.00146× dense (95% document-bootstrap interval: 1.00131–1.00161); the 4096–8191-token ratio was 1.00201. On an untouched 2000-case semantic test, typed execution was 1998/2000, versus 374/2000 for a capacity-matched untyped graph. Two fresh adaptation/selection runs reproduced both directions, although every matched-capacity marginal graph-effect interval included zero. Q25 retained 78.125% of token-KV bytes and 98.44% of dense full-sequence throughput; a tested Q30 export failed. Thus, interaction-aware localization preserved 8192-token language-model quality, while a separate typed path controlled event-role behavior. The experiment does not establish that graph computation caused perplexity preservation.

Machine Learning and Knowledge ExtractionVol. 8(9)
Al-Farabi Kazakh National University (KZ)
Openalex Percentile: Top 8%
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.