Enhancing Diagnostic Accuracy in Bangla: LLM Approaches to Gender Based Medical Specialty Prediction

Question-answering (QA) is the most important form of communication and knowledge exchange. Medical QA, the first step of diagnosis, is used by medical professionals to find diseases experienced by patients. Recently, leveraging the transformer architecture, the latest machine learning models known as Large Language Models (LLMs) achieved state-of-the-art performance in various tasks including open- and closed-domain QA. LLMs can potentially revolutionize the overstretched healthcare systems of the least developing countries such as Bangladesh. In this paper, we have used various types of prompts to use the knowledge embedded in the billions of LLM parameters to find potential doctors based on the descriptions of the patient’s uneasiness. Then the LLMs outputs are parsed to suggest the relevant categories of doctors. The performance of the LLM was heavily dependent on the prompts, the size of the LLM, and portions of the medical data in the pre-training corpus. Based on our experimentation, GPT-4-Turbo demonstrated superior performance compared to other LLMs. Among open-source LLMs, Llama3-8b excelled in zero-shot prompting, while Mistral-7b showed better results in mixed prompting scenarios. However, Llama3-8b was the largest CO <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</inf>, resulting in a significant carbon footprint. In terms of balancing performance with considerations of time and accuracy, Qwen2-7b emerged as the most optimal among open-source models.