EDEL: Enhancing Dense Retrievers for Curation of Biomedical Knowledge Bases

Abstract Motivation Retrieval of relevant papers from the literature is the first step in curating high-quality biomedical knowledge bases. While BM25 has long been the method of choice, dense retrieval models achieved improved accuracy by embedding queries and documents into dense vector representations. Existing knowledge bases provide a natural source for deriving query-document pairs for training such models. Current training approaches, however, do not take into account that some evidence described by knowledge base entries may only be partially expressed in document abstracts, while the full evidence is often contained in inaccessible full texts, introducing noise into binary relevance labels. In addition, existing approaches only make limited use of the knowledge base structure for selecting negative samples during training. Results We propose EDEL, a novel dense bi-encoder for biomedical knowledge base curation to enable curators to find relevant papers for annotation faster. It introduces a loss function using graded relevance scores instead of binary labels to facilitate learning from partially grounded examples, together with a structured sampling strategy that exposes the model to diverse and hard negative examples during training. We evaluate EDEL’s performance in two curation settings, namely precision oncology (on CIViC and OncoKB) and post-translational modifications (on UniProt). EDEL outperforms other state-of-the-art models in NDCG@10 by 1.5 and 3.4 percentage points, respectively. Ablation studies show the effectiveness of both innovations. These results indicate that EDEL can substantially improve literature retrieval for biomedical knowledge base curation. Availability and implementation Code to reproduce our results is available at: https://github.com/WangXII/edel_repo. Supplementary information Supplementary data is attached to this manuscript.