MedSure AI: A Multi-Agent Clinical Governance Framework for Multilingual Healthcare Triage Using Hybrid RRF Retrieval and Claim Verification
Deploying Large Language Models (LLMs) in patient-facing clinical environments poses severe safety hazards, including unverified factual hallucinations, retrieval degradation across non-English and code-mixed queries, failure to escalate acute medical emergencies, and blindness to patient-specific drug contraindications. To address these limitations, we present MedSure AI, an end-to-end multi-agent clinical governance framework that decouples generative synthesis from deterministic and evidence-based safety verification. Operating across English, Hindi, Marathi, and Romanized Hinglish, MedSure AI integrates five specialized components: (1) a script-aware Language Agent and bilingual clinical lexicon mapper; (2) a deterministic Pre-Retrieval Emergency and Risk Triage Engine that short-circuits life-threatening presentations in <10 ms; (3) a Hybrid Reciprocal Rank Fusion (RRF) retriever combining sparse BM25 lexical matching with dense intfloat/multilingual-e5-small vector embeddings over 211 curated World Health Organization (WHO) and Indian Council of Medical Research (ICMR) Standard Treatment Workflows (2,148 verified clinical chunks); (4) a Tier-2 Evidence-Guided Clinical Fallback Synthesizer coupled with a Longitudinal Electronic Health Vault (incorporating multimodal OCR lab report extraction, PBKDF2-HMAC-SHA256 security, and deterministic pharmacovigilance contraindication guards); and (5) an automated Claim Verification and Policy Decision Engine enforcing five discrete clinical actions (ANSWER, REVISE, CLARIFY, ABSTAIN, and ESCALATE) backed by 75 automated unit tests. Evaluated against unconstrained LLM (llm_only) and naive RAG (rag_only) baselines on MedSureBench-Clean (N=150), MedSure AI achieves 100.0% action accuracy, 100.0% emergency recall (65/65), a 0.0% unsafe-answer rate, and 0.0% over-refusals (0/85). Furthermore, stress-testing across the expanded 18-specialty MedSureBench-300 (N=300) benchmark under upstream HTTP 429 API quota exhaustion demonstrates 60.0% offline extractive action accuracy, 66.7% overall emergency escalation recall (80/120), and a 0.0% over-refusal rate (0/170), establishing a fault-tolerant, auditable paradigm for safe multilingual healthcare AI.
Authors
- Mayuri Jagtap
- Jayant Deshmukh
- Shruti Bodkhe
- Ashwini Shahapurkar
- Shreya Madne
Institutions
- MIT Art, Design and Technology University (IN)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-17
- DOI
- https://doi.org/10.5281/zenodo.24323808
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- preprint