Medical multimodal large language models: A survey

• • Provide a detailed analysis of the tasks, architecture, optimization methods, and prompt enhancement techniques of medical MLLMs. • • Offer a comprehensive introduction to the training methods and stages of medical MLLMs, categorizing datasets based on different training phases. • • Discuss two advanced training strategies for activating the reasoning ability of medical MLLMs: supervised reasoning fine-tuning and deep reinforcement learning. • • Summarize the key challenges currently faced by medical MLLMs and propose potential solutions. In recent years, multimodal large language models (MLLMs) have gradually given rise to medical multimodal large language models (Medical MLLMs) through the integration of multimodal data such as clinical reports, medical images, physiological signals, and doctor–patient conversations. This progress has substantially improved the efficiency and quality of clinical question answering. Given the rapid development of this field and its broad clinical potential, this survey presents a systematic review of the core tasks, fundamental principles, methodological innovations, and future research directions of Medical MLLMs. Specifically, this survey first outlines the core medical tasks addressed by Medical MLLMs. It then dissects the three key modules of Medical MLLMs and, through a fine-grained vector-level analysis, explains how input feature vectors are processed and propagated across these modules. Subsequently, existing medical datasets are systematically categorized according to different stages of model training to help researchers identify relevant resources more efficiently. Furthermore, this survey elaborates on the training and evaluation methods of Medical MLLMs and discusses advanced strategies for activating their reasoning capabilities. Finally, it summarizes their practical applications in medical scenarios and highlights the critical challenges and future directions. This survey aims to provide a comprehensive reference for researchers in the field of Medical MLLMs and to promote the adaptation of Medical MLLMs to increasingly complex and diverse medical tasks.

Authors

Institutions

Publication Details

Journal
Information Fusion
Published
2026-04-16
DOI
https://doi.org/10.1016/j.inffus.2026.104386
Primary Topic
Machine Learning in Healthcare
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Medical multimodal large language models: A survey

Hanguang Xiao, Yong Zhong Xu, Jincheng Peng, Zhipeng Li et al.
Information Fusion
Machine Learning in Healthcare
article

Medical multimodal large language models: A survey

Hanguang Xiao, Yong Zhong Xu, Jincheng Peng, Zhipeng Li, Ningzhi Hui
article en

Abstract

• • Provide a detailed analysis of the tasks, architecture, optimization methods, and prompt enhancement techniques of medical MLLMs. • • Offer a comprehensive introduction to the training methods and stages of medical MLLMs, categorizing datasets based on different training phases. • • Discuss two advanced training strategies for activating the reasoning ability of medical MLLMs: supervised reasoning fine-tuning and deep reinforcement learning. • • Summarize the key challenges currently faced by medical MLLMs and propose potential solutions. In recent years, multimodal large language models (MLLMs) have gradually given rise to medical multimodal large language models (Medical MLLMs) through the integration of multimodal data such as clinical reports, medical images, physiological signals, and doctor–patient conversations. This progress has substantially improved the efficiency and quality of clinical question answering. Given the rapid development of this field and its broad clinical potential, this survey presents a systematic review of the core tasks, fundamental principles, methodological innovations, and future research directions of Medical MLLMs. Specifically, this survey first outlines the core medical tasks addressed by Medical MLLMs. It then dissects the three key modules of Medical MLLMs and, through a fine-grained vector-level analysis, explains how input feature vectors are processed and propagated across these modules. Subsequently, existing medical datasets are systematically categorized according to different stages of model training to help researchers identify relevant resources more efficiently. Furthermore, this survey elaborates on the training and evaluation methods of Medical MLLMs and discusses advanced strategies for activating their reasoning capabilities. Finally, it summarizes their practical applications in medical scenarios and highlights the critical challenges and future directions. This survey aims to provide a comprehensive reference for researchers in the field of Medical MLLMs and to promote the adaptation of Medical MLLMs to increasingly complex and diverse medical tasks.

Information FusionVol. 134
Shenzhen Institute of Information Technology (CN), Harbin Institute of Technology (CN), Chongqing University of Technology (CN)
National Natural Science Foundation of China, Natural Science Foundation Project of Chongqing, Chongqing Science and Technology Commission
Quality Education
Openalex Percentile: Top 15%
Machine Learning in Healthcare
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.