Analysis and Evaluation of Open Source Scientific Entity Datasets

Focusing on the construction and application of research entity datasets based on scientific literature, it emphasizes the crucial role of data quality in enhancing natural language processing applications in scientific literature mining, addressing the gaps in detailed descriptions and evaluations of datasets in existing research, and promoting advancements in scientific literature entity recognition technology. Through extensive online research and literature review, 22 open-source research entity datasets were analyzed, focusing on the dataset lifecycle’s collection, annotation, release, and application stages. A set of quality assessment methods for research entity datasets is proposed, along with a discussion of the challenges in dataset construction and strategies for efficient use. The study emphasizes the importance of high-quality datasets for natural language processing applications, proposing evaluation methods and sharing strategies intended to serve as a reference for constructing, selecting, and using scientific literature entity datasets.