Exploiting Entity Relation Extraction via Pre-trained Language Model BERT

Entity relation extraction (ERE) plays a pivotal role in converting unstructured text into structured knowledge triplets. However, the recent works based on the contextual semantic learning usually apply a static context embeddings to represent each token in texts, whereas ignoring the semantic correlations among tokens. To overcome this problem, this paper proposes a novel framework leveraging the bidirectional encoder representations from transformers (BERT) to address challenges in the ERE task, particularly focusing on capturing contextual semantics and resolving complex entity relationships. In specific, the core innovation lies in threefold enhancements. First of all, a context semantic-aware input encoding strategy is designed by explicitly marking entity boundaries using special tokens (e.g., [ENTITY1] and [ENTITY2]). This setting can facilitate BERT to focus on relational semantics while preserving positional information. Subsequently, a LSTM layer and an attention layer are employed to re-conduct the semantic fusion operation by concatenating the learned local token embedding and the global sentence ones. Besides, this paper also incorporates multi-level feature fusion by combining the above semantic fusion embeddings with entity-specific average pooling vectors, effectively balancing context and local entity interactions. Finally, we introduce a relation classifier during fine-tuning to judge whether the given entity pair in this sentence should be attribute which relation type. According to the designed model, we conduct a series of experiments on a general entity relation extraction dataset called NYT. The final experiments on benchmark datasets demonstrate that the proposed model can achieve state-of-the-art performance, outperforming existing approaches that utilize the traditional static token embeddings.