Knowledge Enhanced Language Model for Biomedical Natural Language Processing: Introducing a New Language Model for BioNLP

Following the success of pre-trained language models (PLMs), the biomedical research community has presented various domain-specific PLMs trained on a large biomedical and clinical corpus for biomedical natural language processing (BioNLP) tasks. Those domain-specific PLMs have improved their performance; however, pre-training is time-consuming and computationally expensive, making it unacceptable to most users. Further, those domain-specific PLMs may require additional knowledge to make inferences like experts with relevant knowledge would make. For machines to achieve this capability, we present a knowledge-enhanced language model (KeLM) to enhance the language representation that injects x, y, z triples as domain knowledge into the biomedical text. KeLM enhances the LMs by injecting knowledge without requiring pre-training because KeLM can load the model parameters of any well-trained PLMs. Experimental evaluation on three downstream BioNLP tasks shows that enhancing PLMs with knowledge (KeLM) outperforms various general and domain-specific PLMs on all three downstream BioNLP tasks.