CMiNER: Named entity recognition on imperfectly annotated data via confidence and meta weight adaptation

Named Entity Recognition(NER) is a very critical task in natural language processing . The task often suffers from noisy labels that affect the robustness of the model. In particular, when there are a large number of both mislabeled and missing labeled labels in the dataset, the performance of the model is usually severely degraded. Through empirical studies of multiple NER models and datasets, we find that the degree of degradation increases with the noise rate, which poses a challenge to robustly training models on imperfectly annotated datasets. To alleviate the problem, this study designs a confidence and meta-weight based adaptive NER training framework (CMiNER). We design entity-level confidence estimators that not only initially assess the quality of a given noisy dataset, but also greatly assist in the training phase. Specifically, we design a meta-weight adaptive strategy that incorporates confidence partitioning for robust training of noisy datasets. We partition the data into support and query sets using a confidence estimator and employ a meta-weight adaptation strategy for training, ultimately enhancing the model performance. Experiments on real NER datasets and synthetic noisy datasets show that the CMiNER framework is extremely robust. The performance after introducing our framework on the NER datasets outperforms other noise reduction methods. The performance is improved by an average of 10% on low-noise synthetic datasets and 3.1% on high-noise synthetic datasets. In addition, there is a 7.6% improvement over the real NER dataset. • CMiNER improves robustness without reducing noisy data. • CMiNER is useful for solving both mislabeling and missing label problems. • CMiNER improves the performance on both noisy and real datasets. • Multiple experiments validate that NER model suffers severely from noisy data.