Evaluating Few-Shot Prompting Approach Using GPT4 in Comparison to BERT-Variant Language Models in Biomedical Named Entity Recognition

The wealth of information associated with the exponential increase in digital text, particularly within the biomedical field, has the potential to advance medical research, improve patient care, and enhance public health outcomes. However, the sheer volume and complexity of this data necessitate advanced computational tools for effective processing and analysis. We investigated the use of various pretrained transformer-based language models, particularly BERT, PubMedBERT, SciBERT, ClinicalBERT, DistilBERT, and the application of prompt engineering with GPT-4, within the context of biomedical Named Entity Recognition. Our approach incorporates a comprehensive performance evaluation analysis utilizing standard NLP evaluation metrics and computational resource usage metrics such as training time, memory usage, and inference time. Through this multifaceted approach, we sought to find out how the few-shot prompting approach using GPT4 performs in comparison to the BERT-variant language models while at the same time identifying models that not only excel in performance efficiency but also demonstrate computational affordability. Our experimental results show that even the most basic transformer-based language model outperforms the few-shot prompting approach of GPT-4, despite the popularity of the LLM in the more general Natural Language Processing tasks.