Re-SciBERT: An Entity-Enriched Language Model to Enhance Biomedical Relation Extraction

Relation extraction is a key task in biomedical natural language processing (NLP), aiming to identify and extract relationships between entities across various sources, including clinical texts, research papers, and other biomedical literature. Recently, transformer-based models have set new benchmarks across various NLP tasks. However, relation extraction poses unique challenges, as it depends on understanding both the broader context and the specific interactions between targeted entities. In this paper, we introduce RE-SciBERT, an entityenriched language model that leverages a modified SciBERT architecture, enhanced with entity embeddings, to capture nuanced relationships between biomedical entities. By integrating explicit entity information directly into the transformer model, we aim to improve the F1 scores of relation extraction tasks compared to standard transformer-based methods. We evaluate our model on benchmark datasets, demonstrating that the entity-enrichment strategy significantly boosts performance, particularly in scenarios involving complex or ambiguous entity relations. The results highlight the potential of transformer-based models with specialized entity embeddings in advancing the task of biomedical relation extraction.