LGENER: A lattice- and GAN-based method for Chinese ethnic NER

Named entity recognition (NER) aims to find target entities from a given unstructured text and determine their category. Limited annotated data and the presence of numerous complex entities characterize the task of named entity recognition in ancient ethnic texts. Existing NER methods have failed to consider boundary information in ancient ethnic texts and to address issues such as polysemy and long-distance dependencies within these texts. To address these issues effectively, this paper proposes an ethnic named entity recognition method that integrates a lattice and generative adversarial network (LGENER). First, a generator model based on BERT and Flat is designed and incorporates an auxiliary task for boundary detection of named entities. Second, a discriminator based on a CNN is designed, introducing a model parameter update strategy that calculates the consistency of the feature distributions between generated and real labels. Ultimately, the model is able to increase the accuracy of entity category label generation through adversarial training on low-resource datasets. The LGENER model achieves promising results on the dataset of Yi ethnic ancient text named entities, with F1 scores, precision, and recalls reaching 86.86 %, 86.27 %, and 87.45 %, respectively, when the loss weight coefficient β for the auxiliary task is set to 0.5. Experiments on the Bai ethnic dataset with the same labeling system and similar text styles and other general Chinese datasets, such as Resume, also demonstrate the effectiveness of the proposed method.