A Framework for automated selective Fine-Tuning of Domain-Specific Large Language Models Using Graph-Based Retrieval Augmented Generation

Graph based retrieval augmented generation technique in Large Language Model (LLM) brings in major advantages by providing deep context to LLMs through relational knowledge graph for text generation, classification, question and answering use cases. However, maintaining vast data volume of domain specific data in a knowledge graph with complex relationships and querying from it every time a prompt is being posted to LLM, is a time consuming and expensive process. This paper presents a novel framework for selectively fine-tuning domain-specific large language models (LLMs) using a multi-stage Knowledge Graph (KG) based Retrieval Augmented Generation (RAG) pipeline and an Automated Incremental Fine-tuning System (AIFS). The proposed system aims to enhance the accuracy and relevance of LLM responses for text generation and Question Answering use cases by finetuning the LLM incrementally based on highly sought and highly relevant information in knowledge graph identified by leveraging page rank algorithm in KG. The framework comprises three major subsystems: Knowledge Graph Generation, Automated Incremental fine-tuning system (AIFS), and Domain Based Information Retrieval (DBIR). The effectiveness of the system is demonstrated through its ability to incrementally fine-tune LLMs based on selected highly relevant nodes within the KG, thereby improving the model’s domain-specific knowledge, response accuracy by 90% and reduce cost by 71.8%.