MedSure AI: A Multi-Agent Clinical Governance Framework for Multilingual Healthcare Triage Using Hybrid RRF Retrieval and Claim Verification

Deploying Large Language Models (LLMs) in patient-facing clinical environments poses severe safety hazards, including unverified factual hallucinations, retrieval degradation across non-English and code-mixed queries, failure to escalate acute medical emergencies, and blindness to patient-specific drug contraindications. To address these limitations, we present MedSure AI, an end-to-end multi-agent clinical governance framework that decouples generative synthesis from deterministic and evidence-based safety verification. Operating across English, Hindi, Marathi, and Romanized Hinglish, MedSure AI integrates five specialized components: (1) a script-aware Language Agent and bilingual clinical lexicon mapper; (2) a deterministic Pre-Retrieval Emergency and Risk Triage Engine that short-circuits life-threatening presentations in <10 ms; (3) a Hybrid Reciprocal Rank Fusion (RRF) retriever combining sparse BM25 lexical matching with dense intfloat/multilingual-e5-small vector embeddings over 211 curated World Health Organization (WHO) and Indian Council of Medical Research (ICMR) Standard Treatment Workflows (2,148 verified clinical chunks); (4) a Tier-2 Evidence-Guided Clinical Fallback Synthesizer coupled with a Longitudinal Electronic Health Vault (incorporating multimodal OCR lab report extraction, PBKDF2-HMAC-SHA256 security, and deterministic pharmacovigilance contraindication guards); and (5) an automated Claim Verification and Policy Decision Engine enforcing five discrete clinical actions (ANSWER, REVISE, CLARIFY, ABSTAIN, and ESCALATE) backed by 75 automated unit tests. Evaluated against unconstrained LLM (llm_only) and naive RAG (rag_only) baselines on MedSureBench-Clean (N=150), MedSure AI achieves 100.0% action accuracy, 100.0% emergency recall (65/65), a 0.0% unsafe-answer rate, and 0.0% over-refusals (0/85). Furthermore, stress-testing across the expanded 18-specialty MedSureBench-300 (N=300) benchmark under upstream HTTP 429 API quota exhaustion demonstrates 60.0% offline extractive action accuracy, 66.7% overall emergency escalation recall (80/120), and a 0.0% over-refusal rate (0/170), establishing a fault-tolerant, auditable paradigm for safe multilingual healthcare AI.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-17
DOI
https://doi.org/10.5281/zenodo.24323808
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

MedSure AI: A Multi-Agent Clinical Governance Framework for Multilingual Healthcare Triage Using Hybrid RRF Retrieval and Claim Verification

Mayuri Jagtap, Jayant Deshmukh, Shruti Bodkhe, Ashwini Shahapurkar et al.
2 links
Zenodo (CERN European Organization for Nuclear Research)
Artificial Intelligence in Healthcare and Education
preprint

MedSure AI: A Multi-Agent Clinical Governance Framework for Multilingual Healthcare Triage Using Hybrid RRF Retrieval and Claim Verification

Mayuri Jagtap, Jayant Deshmukh, Shruti Bodkhe, Ashwini Shahapurkar, Shreya Madne
preprint en

Abstract

Deploying Large Language Models (LLMs) in patient-facing clinical environments poses severe safety hazards, including unverified factual hallucinations, retrieval degradation across non-English and code-mixed queries, failure to escalate acute medical emergencies, and blindness to patient-specific drug contraindications. To address these limitations, we present MedSure AI, an end-to-end multi-agent clinical governance framework that decouples generative synthesis from deterministic and evidence-based safety verification. Operating across English, Hindi, Marathi, and Romanized Hinglish, MedSure AI integrates five specialized components: (1) a script-aware Language Agent and bilingual clinical lexicon mapper; (2) a deterministic Pre-Retrieval Emergency and Risk Triage Engine that short-circuits life-threatening presentations in <10 ms; (3) a Hybrid Reciprocal Rank Fusion (RRF) retriever combining sparse BM25 lexical matching with dense intfloat/multilingual-e5-small vector embeddings over 211 curated World Health Organization (WHO) and Indian Council of Medical Research (ICMR) Standard Treatment Workflows (2,148 verified clinical chunks); (4) a Tier-2 Evidence-Guided Clinical Fallback Synthesizer coupled with a Longitudinal Electronic Health Vault (incorporating multimodal OCR lab report extraction, PBKDF2-HMAC-SHA256 security, and deterministic pharmacovigilance contraindication guards); and (5) an automated Claim Verification and Policy Decision Engine enforcing five discrete clinical actions (ANSWER, REVISE, CLARIFY, ABSTAIN, and ESCALATE) backed by 75 automated unit tests. Evaluated against unconstrained LLM (llm_only) and naive RAG (rag_only) baselines on MedSureBench-Clean (N=150), MedSure AI achieves 100.0% action accuracy, 100.0% emergency recall (65/65), a 0.0% unsafe-answer rate, and 0.0% over-refusals (0/85). Furthermore, stress-testing across the expanded 18-specialty MedSureBench-300 (N=300) benchmark under upstream HTTP 429 API quota exhaustion demonstrates 60.0% offline extractive action accuracy, 66.7% overall emergency escalation recall (80/120), and a 0.0% over-refusal rate (0/170), establishing a fault-tolerant, auditable paradigm for safe multilingual healthcare AI.

Zenodo (CERN European Organization for Nuclear Research)
MIT Art, Design and Technology University (IN)
Good health and well-being
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon