Winning DocVQA 2026: Multi-Agent Orchestration for Multi-Domain Document Reasoning

Abstract Document Visual Question Answering (DocVQA) over heterogeneous domains requires precise extraction and reasoning over documents whose formats range from business reports and scientific papers to maps, comics, and engineering drawings. No single vision-language model (VLM) performs uniformly well across such diverse content. We present a multi-agent document-reasoning system submitted to the DocVQA 2026 competition at ICDAR 2026, where it placed first in Task 1 (General category, \((>)\)35B-parameter class) with 60.00% ANLS (95% Wilson-score CI: \((\pm)\)7 pp; \((n = 160)\)), 13.75 pp ahead of the second-placed system (46.25%). The system combines three components: (1) IDA, a layout-aware OCR engine that produces structured Markdown output as a deterministic text anchor; (2) four frontier VLMs (Gemini 3.1 Pro, Gemini 2.5 Pro, Claude Sonnet 4, Qwen3.5) performing question-guided focused reads with model-adapted question reformulations; and (3) a Claude Opus 4.6 reasoning agent that synthesizes perspectives through domain-specific trust hierarchies and cross-perspective conflict detection. A first submission reached 55.63% ANLS; a refinement pass using the first run's reasoning outputs as prior context lifted the score to 60.00% (\((+4.37)\) pp). The system improves over the strongest single-model zero-shot baseline on the leaderboard (Gemini 3.1 Pro: 37.50%) by \((+22.5)\) pp and over a naive mixture-of-experts estimate (\((\sim)\)40%) by approximately \((+20)\) pp. We attribute roughly \((+7)\) pp to IDA's deterministic text extraction and \((+13)\) pp to multi-agent orchestration; these are attributed estimates from internal experiments rather than controlled-ablation measurements, and we state this limitation explicitly. Per-domain, the system scores highest on text-heavy documents (business reports 0.75, scientific papers 0.70 ANLS), where deterministic extraction anchors the answer, and lowest on comics (0.45), with maps and science posters at 0.50.

Authors

Publication Details

Journal
Research Square
Published
2026-11-13
DOI
https://doi.org/10.21203/rs.3.rs-11103268/v1
Primary Topic
Handwritten Text Recognition Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Winning DocVQA 2026: Multi-Agent Orchestration for Multi-Domain Document Reasoning

Welf Wustlich
1 links
Research Square
Handwritten Text Recognition Techniques
preprint

Winning DocVQA 2026: Multi-Agent Orchestration for Multi-Domain Document Reasoning

Welf Wustlich
preprint en

Abstract

Abstract Document Visual Question Answering (DocVQA) over heterogeneous domains requires precise extraction and reasoning over documents whose formats range from business reports and scientific papers to maps, comics, and engineering drawings. No single vision-language model (VLM) performs uniformly well across such diverse content. We present a multi-agent document-reasoning system submitted to the DocVQA 2026 competition at ICDAR 2026, where it placed first in Task 1 (General category, \((>)\)35B-parameter class) with 60.00% ANLS (95% Wilson-score CI: \((\pm)\)7 pp; \((n = 160)\)), 13.75 pp ahead of the second-placed system (46.25%). The system combines three components: (1) IDA, a layout-aware OCR engine that produces structured Markdown output as a deterministic text anchor; (2) four frontier VLMs (Gemini 3.1 Pro, Gemini 2.5 Pro, Claude Sonnet 4, Qwen3.5) performing question-guided focused reads with model-adapted question reformulations; and (3) a Claude Opus 4.6 reasoning agent that synthesizes perspectives through domain-specific trust hierarchies and cross-perspective conflict detection. A first submission reached 55.63% ANLS; a refinement pass using the first run's reasoning outputs as prior context lifted the score to 60.00% (\((+4.37)\) pp). The system improves over the strongest single-model zero-shot baseline on the leaderboard (Gemini 3.1 Pro: 37.50%) by \((+22.5)\) pp and over a naive mixture-of-experts estimate (\((\sim)\)40%) by approximately \((+20)\) pp. We attribute roughly \((+7)\) pp to IDA's deterministic text extraction and \((+13)\) pp to multi-agent orchestration; these are attributed estimates from internal experiments rather than controlled-ablation measurements, and we state this limitation explicitly. Per-domain, the system scores highest on text-heavy documents (business reports 0.75, scientific papers 0.70 ANLS), where deterministic extraction anchors the answer, and lowest on comics (0.45), with maps and science posters at 0.50.

Research Square
Handwritten Text Recognition Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon