AI Chatbot for Cancer Patient Support: Development and Evaluation Using Llama 3.1, Mistral 7B, and PHI 3B

This study develops and evaluates a question-answering (Q&A) system for breast cancer patients using generative AI technologies. We compared the performance of three language models-Llama 3.1, Mistral 7B and Phi 3.5. The goal is to integrate this system into a patient-facing application, providing personalised, interactive support based on reliable sources such as Cancer Research UK. However, findings indicate that misinformation remains a significant concern. Medical chatbots utilising retrieval augmented generation (RAG) providing clinical information require significant refinement before being suitable for clinical use.