Advancing Arabic Medical Question Answering Systems with RAG and LLMs Integration

The integration of large language models (LLMs) in healthcare represents a crucial advancement in utilizing cutting-edge technology for patient care and medical research. The main problem with these models is their imperfect accuracy and propensity for producing hallucinations-responses that are factually incorrect or irrelevant-posing risks in medical applications. This study addresses these limitations by enhancing the performance of medical question-answering (QA) systems through the integration of the Retrieval-Augmented Generation (RAG) technique with LLMs. Since our focus was on Arabic, we selected the JAIS, the most recent LLM designed to support Arabic. The experiments done using the jais-13b LLM, evaluating its base performance, fine-tuned version, and RAG-enhanced implementation on an Arabic medical QA dataset. The base model attaining 95.6% accuracy and an F-measure of 90.4%, which improved to 97.8% accuracy and 93.7% F-measure after fine-tuning. Integration RAG further improved the results, attaining 98.7% accuracy and 97.6% F-measure. Furthermore, integrating RAG strategies with the JAIS LLM outperformed previous studies in terms of accuracy and F-score. These findings highlight the potential of RAG in mitigating hallucinations and enhancing factual consistency in Arabic medical QA system.