BERT-Based Semantic Relations Extraction from Large-scale Medical Datasets

Relation Extraction (RE) is a crucial task which aims to identify and classify relations between entities in a given text, particularly, in the case of medical data, where understanding the relations between entities is essential for knowledge extraction. In the literature, Deep Learning (DL) models like BERT have shown promising results in RE tasks. Nevertheless, most of the proposed approaches do not incorporate pretraining before conducting finetuning on a new task, which could considerably improve performance. In this paper we present a novel deep learning-based approach for extracting semantic relations from unstructured medical datasets. Our approach takes place in three main phases: (1) a pretraining phase using unsupervised relation extraction techniques, (2) a fine-tuning phase employing supervised relation extraction, and (3) an inference phase for semantic relations’ deduction. Our relation extraction method is based on combining both unsupervised RE (during the pretraining phase) and supervised RE (during the fine-tuning phase). Our three phases-based approach, includes the hybridization of different BERT variants. The evaluation was carried out on the SemEval 2010 dataset. Experimental results show improvements in the evaluation metrics, which confirms the great interest of our proposal.