ChatEndoscopist: A Domain-Specific Chatbot with Images for Gastrointestinal Diseases

This study aims to enhance domain-specific medical knowledge within large language models (LLMs) by developing a chatbot, chatEndoscopist, a specialized model for oesophageal cancer. In particular, the chatbot incorporates related images to further elucidate the retrieved content while providing answers. Fine-tuned BioMistral LLM with 50 related documents, a dataset specifically curated for medical literature, ChatEndoscopist was compared to ChatGPT. For text answers, despite its specialized training, ChatGPT appears to outperform ChatEndoscopist in precision (0.210 vs. 0.148), recall (0.323 vs. 0.049), and F1 score (0.266 vs. 0.099). ChatGPT also demonstrated superior lexical diversity with a Type-Token Ratio (TTR) of 0.772 and Lexical Density of 0.813, compared to ChatEndoscopist's TTR of 0.717 and Lexical Density of 0.781. This in part, could be due to the limited documents to fine tune. However, the related images are mostly retrieval with regarding to user's queries. Future work will focus on incorporating more related papers to balance specialized accuracy with broader linguistic flexibility.