Token Graph Generation for Robust Named Entity Recognition

Named Entity Recognition (NER) is to extract pieces from text and classify them into predefined categories.For most state-of-the-art NER methods, the attention mechanism is a core component of their network architectures.Meanwhile, it is often blamed for introducing of spurious token correlations, which do harm to the prediction accuracy.To make the NER model robust to such noisy token relations, we focus on how to remove these relations from the complete token graph and identify the actually useful token graph for NER.Starting with a complete token graph from the self-attention matrix, we propose a token graph generated method to explicitly model the core token graph construction process by deleting edges iteratively in a reinforcement learning framework.The Generated Token Graph, denoted as GTG, can be utilized in two primary ways: as a masking matrix to serve as an adapter for attention mechanism, namely GTG-adapter.Experimental results on NER benchmark datasets show our proposed method will improve the accuracy and robustness of NER models compared with SOTA baselines.