Reproducibility in Named Entity Recognition: A Case Study Analysis

Information extraction from text is essential in data science and artificial intelligence, especially with the increase in the number of scientific articles. Robust methods are needed to structure texts and highlight key information. Named Entity Recognition (NER) identifies and classifies major elements in text, aiding in dataset structuring and tagging.Under the FAIRClinical and ShareFAIR projects, we aimed to extract information from clinical trial publications using NER models. These models have been successfully used for clinical trial information extraction. In this paper, we report on the difficulties met in reusing existing NER solutions, focusing on mandatory replicability. This paper discusses the challenges faced in replicating a notable study, the difficulties encountered and lessons learned. It compares our experiment with the feedback provided by the literature and draws conclusions.