Towards Instruction-Tuned Verification for Improving Biomedical Information Extraction with Large Language Models

Biomedical information extraction, including tasks like Named Entity Recognition (NER) and Relationship Extraction (RE), plays a critical role in analyzing unstructured biomedical data, facilitating more accurate downstream applications such as clinical decision support and personalized medicine. Although Large Language Models (LLMs) excel in various domains, they often struggle with issues like hallucinations and domain-specific inaccuracies, limiting their effectiveness in biomedical applications. To tackle these challenges, we propose a novel framework that incorporates an external dual-stage verifier to enhance biomedical information extraction. This verifier is prompted to first identify and complete missing entities and relations, and then filter out incorrect extractions, thereby enhancing both precision and recall in information extraction. Furthermore, we proposed an approach for constructing the instruction-tuning dataset using both positive and negative demonstrations, which improves the verifier’s correction capabilities through Parameter-Efficient Fine-Tuning (PEFT). Our extensive experiments on three public benchmark datasets cover both biomedical NER and RE tasks, comparing four LLMs ranging from 7 to 175 billion parameters to validate the effectiveness and scalability of our approach. The results demonstrate that by integrating an external verifier specifically finetuned for biomedical tasks, we achieve substantial improvements in precision, recall, and F1 scores for NER and RE tasks. Our method consistently outperforms In-Context Learning (ICL) approaches in F1 scores, achieving increases of up to 20% in some cases. Furthermore, ablation studies demonstrate the effectiveness of both stages in the verification process. The proposed framework provides valuable insights for accurate biomedical information extraction using LLMs, highlighting its potential applications in real-world clinical settings.