Enhancing medical data processing with RAG-augmented fine-tuned LLMs and guardrails

As the requirement grows in finding more and more reliable and trustworthy medical query processing chatbots that are based on LLM should be well-designed to ensure accuracy of responses as well as their ethicality. Our approach considers a lightweight fine-tuning model based on Quantized Low-Rank Adaptation (QLoRA) to speedup the adaptation of the model on domain-specific medical data on scarce computational resources. NeMo-Guardrails are also part of the system to ensure the ethical compliance and safety which ensure one to respond to the content getting hallucination free. We choose to use a parameter model which is fine-tuned using domain specific data, gemmaq1.1-7B, which give clinically relevant and context aware outputs. The use of Retrieval-Augmented Generation (RAG) is used to bring the external medical context dynamically. Embeddings called query embeddings are created by the pubmedbert-base-embeddings model which is built on the BioBERT architecture and trained on medical text. The embeddings are stored in a Qdrant vector data base and searched using cosine similarity, best results are added to the prompt to increase the accuracy and factual basis. The metrics of NLP like BLEU, ROUGE-L, BERTscore and QAG were used for the evaluation of the method suggested. The study helps to develop safe, grounded and clinically robust LLM based medical chatbot which can be utilized in the real health care settings.