Textual data augmentation using generative approaches - Impact on named entity recognition tasks

Industrial applications of Named Entity Recognition (NER) are usually confronted with small and imbalanced corpora. This could harm the performance of trained and finetuned recognition models, especially when they encounter unknown data. In this study we develop three generation-based data enrichment approaches, in order to increase the number of examples of underrepresented entities. We compare the impact of enriched corpora on NER models, using both non-contextual (fastText) and contextual (Bert-like) embedding models to provide discriminant features to a biLSTM-CRF used as an entity classifier. The approach is evaluated on a contract renewal detection task applied to a corpus of calls for tenders. The results show that the proposed data enrichment procedure effectively improves the NER model’s effectiveness when applied on both known and unknown data. • We address named-entity recognition tasks in the scope of limited resources and study the impact of data augmentation. • We propose three data augmentation approaches, one based on textual operations using regular expressions and two on generative language models, among which two are put to further evaluation. • We evaluated two retained approaches on an industrial task and measured the impact of data augmentation on the performance of various kinds of recognition models. • We further evaluate a selection of best model combinations regarding their robustness when applied to an unknown corpus.