Attention routing in vision-language encoders: technical note

Technical note recording the method and results, as of 14 November 2026, of a study of attention routing in the image encoders of pretrained vision-language models: the attention pattern of these encoders can be decided from a coarse (downsampled or fixed-size thumbnail) view of the image while content is read at full resolution; each query needs a small, fixed budget of keys chosen by the coarse view; and a coarse-routed block-sparse encoder scales near-linearly with resolution (3.9-4.3x faster than the unmodified encoder at about 50k patches on an A100) at matched accuracy on DocVQA, InfoVQA, TextVQA and ChartQA. SHA-256 of the deposited PDF (technical_note.pdf): da07771593364454c4d652fbbb76a213dbacab1876d13975ba2ab0f998985e1c

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-14
DOI
https://doi.org/10.5281/zenodo.24217417
Primary Topic
Multimodal Machine Learning Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Attention routing in vision-language encoders: technical note

Srinivas Harish
Zenodo (CERN European Organization for Nuclear Research)
Multimodal Machine Learning Applications
article

Attention routing in vision-language encoders: technical note

Srinivas Harish
article en

Abstract

Technical note recording the method and results, as of 14 November 2026, of a study of attention routing in the image encoders of pretrained vision-language models: the attention pattern of these encoders can be decided from a coarse (downsampled or fixed-size thumbnail) view of the image while content is read at full resolution; each query needs a small, fixed budget of keys chosen by the coarse view; and a coarse-routed block-sparse encoder scales near-linearly with resolution (3.9-4.3x faster than the unmodified encoder at about 50k patches on an A100) at matched accuracy on DocVQA, InfoVQA, TextVQA and ChartQA. SHA-256 of the deposited PDF (technical_note.pdf): da07771593364454c4d652fbbb76a213dbacab1876d13975ba2ab0f998985e1c

Zenodo (CERN European Organization for Nuclear Research)
Multimodal Machine Learning Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon