Integrating Open-Source LLMs with Retrieval-Augmented Generation for Obstetrics and Gynecology Domain

In this research paper, we address the integration of Large Language Models (LLM) with Retrieval Augmented Generation (RAG) to enhance clinical decision support and address patient doubts in the Obstetrics and Gynecology (OBGYN) domain. The research mainly tries to explore on How open source LLM’s can effectively retrieve and generate the relevant responses in the OBGYN domain. The research methodology includes two components: Data Ingestion, which reads the input text data and stores it to a vector database in an embedded format and Data Retriever-Generation, which retrieves the relevant information from the vector database and use it for accurate response generation. The method includes data collection from esteemed medical databases, LLM model selection, integration with RAG and evaluation of the generated outputs. The methodology uses Bio-Mistral 7B fine-tuned LLM with PubMed Bert embeddings. The LLM responses are evaluated using the Ragas framework, context precision and context recall to measure the performance of retrieval system, faithfulness to measure hallucinations and answer relevancy to measure how relevant the answers are to the input query. The evaluated results confirm that the research has improved the accuracy as well as the contextual relevancy of the information in the OBGYN domain. This research provides a robust architecture for integration of Artificial Intelligence for supporting clinical decision making and information retrieval in specialized medical domains. This research can be extended with the upcoming advancements in the field of Artificial Intelligence and Data Science.