Zero-Shot Knowledge Extraction with Hierarchical Attention and an Entity-Relationship Transformer

Knowledge extraction from unstructured text is a critical challenge in natural language processing (NLP), especially in zero-shot settings where task-specific labeled data is unavailable. In this paper, we propose a novel zero-shot knowledge extraction framework, ZKET (Zero-shot Knowledge Extraction Transformer), designed to extract structured knowledge from un-structured text without requiring supervision. ZKET integrates a Hierarchical Attention Mechanism (HAM) to capture both local and global dependencies, an Entity-Relationship Transformer (ERT) for extracting entities and their relationships, a Unified Query Generation Module (UQGM) for dynamic query formulation, and an Adaptive Output Schema (AOS) to ensure output consistency with structured knowledge formats. By combining these mechanisms, ZKET significantly improves knowledge extraction performance across various tasks, show-casing enhanced precision, recall, and generalization compared to existing models like GPT-3 and Mistral. This work contributes a flexible, adaptable solution to zero-shot knowledge extraction, advancing the state-of-the-art in unsupervised NLP tasks.