PM3-KIE: A Probabilistic Multi-Task Meta-Model for Document Key Information Extraction

Key Information Extraction (KIE) from visually rich documents is commonly approached as either fine-grained token classification or coarse-grained entity extraction.While tokenlevel models capture spatial and visual cues, entity-level models better represent logical dependencies and align with real-world use cases.We introduce PM3-KIE, a probabilistic multitask meta-model that incorporates both finegrained and coarse-grained models.It serves as a lightweight reasoning layer that jointly predicts entities and all appearances in a document.PM3-KIE incorporates domain-specific schema constraints to enforce logical consistency and integrates large language models for semantic validation, thereby reducing extraction errors.Experiments on two public datasets, DeepForm and FARA, show that PM3-KIE outperforms three state-of-the-art models and a stacked ensemble, achieving a statistically significant 2% improvement in F1 score. IntroductionKey Information Extraction (KIE) focuses on identifying structured key-value pairs from visually rich documents (VRDs) (Huang et al., 2019) based on a predefined schema that specifies the target information types.This capability is essential for automating business document processing across industries such as finance and law.Automating information extraction significantly reduces operational costs: processing a single invoice, for example, can cost $13.11and takes up to eight days (Girsch-Bock, Mary, 2024;Cohen and York, 2020).Despite recent advances, extracting structured data remains challenging for state-of-the-art models, particularly for documents with complex schemas or semi-structured layouts (Wang et al., 2023b).KIE can generally be approached through two distinct paradigms, as illustrated in Figure 1: