Layout-grounded iterative refinement for auditable organization–task extraction in Vietnamese administrative documents

Automatic processing of Vietnamese administrative documents is challenging owing to factors like compact layout, lengthy action clause, and organization list that involve many coordinators. In this paper, we introduce a five-step extraction pipeline using layout for auditing organization-task pairing in scanned administrative documents. Our pipeline involves layout-preserving OCR, contextual grouping, faithful grounding of candidates, task refinement guided by critic, and finally verification of organization. Rather than treating the task as end-to-end multimodal reasoning, the proposed pipeline uses visual and layout information mainly for layout-preserving OCR and then constrains downstream extraction to source-grounded textual evidence. A benchmark of 1,409 documents containing 18,359 pairs of annotations between organizations and tasks is constructed for evaluation purposes. With the primary strict protocol, the whole process yields 94.1% precision, 93.5% recall, and 93.8% F1 scores; under the relaxed protocol, it yields 96.1% F1 score. In order to quantify the benefit over simply prompting, the evaluation setup also contains the controlled baselines of OCR-text pairs using simple chunking, only grouping context formation, and retrieval-based context formation, in addition to the document encoder baseline accounting for the layout information. The grounding-related comparison shows that explicit evidence-window construction alone yields only a modest gain over grouped one-shot extraction, whereas the staged ablation demonstrates that the most substantial benefit is obtained once grounded evidence is coupled with critic-assisted task refinement; the last organization-validation stage handles collaborator-heavy cases not solvable at the first extraction stage.