P²Net: Parallel Pointer-based Network for Key Information Extraction with Complex Layouts

Key Information Extraction (KIE) is a challenging multimodal task aimed at extracting structured value entities from visually rich documents.Despite recent advancements, two major challenges remain.First, existing datasets typically feature fixed layouts and a limited set of entity categories, while current methods are based on a full-shot setting that is difficult to apply in real-world scenarios, where new entity categories frequently emerge.Secondly, current methods often treat key entities simply as parts of the OCR-parsed context, neglecting the positive impact of the relationships between key-value entities.To address the first challenge, we introduce a new large-scale, humanannotated dataset, Complex Layout document for Key Information Extraction (CLEX).Comprising 5,860 images with 1,162 entity categories, CLEX is larger and more complex than existing datasets.It also primarily focuses on the zero-shot and few-shot KIE tasks, which are more aligned with real-world applications.To tackle the second challenge, we propose the Parallel Pointer-based Network (PNet).This model frames KIE as a pointer-based classification task and effectively leverages implicit relationships between key-value entities to enhance extraction.Its parallel extraction mechanism enables simultaneous and efficient extraction of multiple results.Experiments on widely-used datasets, including SROIE, CORD, and the newly introduced CLEX, demonstrate that PNet outperforms existing state-of-the-art methods (including GPT-4V) while maintaining fast inference speeds.