SuSastho.AI: A multimodal medical copilot for adolescents using evidence-based medicine and large language models

Adolescents in Bangladesh face serious sexual, reproductive, and mental health challenges due to cultural stigma, poverty, and limited healthcare infrastructure. Within the country, 63% of adolescents are deprived of essential sexual and reproductive health services, and only 13% receive mental health support. Adolescents living in urban slums and with disabilities face additional challenges in receiving reliable health information. This limited access exposes them to a high risk of sexually transmitted infections (STIs), unintended pregnancies, and serious mental health issues. Addressing these challenges, our study introduces SuSastho.AI, a healthcare copilot providing access to reliable health information to adolescents. We utilized large language models, along with Evidence-Based Medicine, retrieval-augmented generation, and a clinically validated dataset to provide evidence-based responses, supporting both voice and text-based interactions. Clinical evaluation of a pilot study shows our method reduces incorrect responses by 26.9% and increases response correctness by 16.1% compared to other methods. It achieved an accuracy rate of 86.7% when specifically evaluated based on available knowledge. While the responses are mostly consistent with up-to-date medical practices, occasional, less precise responses highlight the need for further refinement. Participants reported overall positive feedback, where 87% found answers to their questions, and 90.7% found responses relevant. Our results show that SuSastho.AI can provide reliable and evidence-based information while being an affordable way to support traditional healthcare systems with a high potential to transform digital health. The study sets an example as an evidence-based healthcare copilot to support adolescents and lays the foundation for future research where evidence-based tools overcome social barriers. • SuSastho.AI is a multimodal evidence-based medical copilot tailored for Bengali-speaking adolescents. • Enhanced clinical accuracy of LLM responses using a novel Chain of Thought reasoning framework and Evidence-Based Medicine. • Achieved 26.9% fewer errors and 16.1% higher correctness compared to the baseline Llama 3 model in a pilot clinical evaluation. • Achieved high user satisfaction, with over 87% finding the answers relevant and actionable. • Addressed ethical AI use in sensitive health interactions, focusing on transparency and privacy.