Medvisiochat: A Multimodal Large Language Model Framework for Interpretable Diagnosis with Visual Grounding in CXRs

Multimodal Large Language Models (MLLMs) have enhanced the interpretability of disease diagnosis, leading to more accurate outcomes and improved patient understanding. A key advancement is visual grounding, linking textual information with medical images like chest x-rays (CXRs) for deeper analysis. This approach combines visual details with expert medical language, enabling MLLMs to detect patterns and disease indicators effectively. However, challenges persist, particularly in multi-category classification and precise visual grounding in CXRs. To address this, we introduce MedVisioChat, an MLLM with visual grounding capabilities for seamless integration of medical images and diagnoses. Pre-trained on large datasets and fine-tuned with specific GPT-4 Turbo instructions, MedVisioChat aims to provide accurate, interactive diagnostics, demonstrating superiority over existing models and promising an inter-pretable visual medical diagnosis.