A Pilot Study on Fine-tuning Named Entity Recognition for Clinical Tag Extraction Using Pretrained Language Models: The Tut-All Experience

The standard procedure to extract data from medical records is still a laborious process where research analysts examine a wide range of information sources. Hence, there is a strong need to improve automatic data extraction techniques. This study performed fine-tuning of the pretrained language models (PLMs)—BioElectra, PubMedBert, BioLinkBert, and Clinical Longformer—for named entity recognition (NER) in clinical texts. The clinical texts were extracted from records using the Tut-All software and were comprised of abstracts and methods sections. For performance analysis the mean of the F1-score for each named entity tag was used across different hyperparameter optimizations. PubMedBert achieved the best performance results with a mean F1-score of 0.638 and an output range of 0.3. This performance was achieved after training with a learning rate and a weight decay of 5×10-5 and 3×10-3 respectively, employing default HuggingFace hyperparameters. Notably, PubMedBert exhibited superior performance on minority entity types, addressing a key challenge in real-world data applications. The optimized model not only expedites information retrieval, but also enhances the overall quality of extracted data, marking a valuable contribution to the data extraction data from medical records.