An LLM-Enabled Multimodal Agentic AI Framework for the Medical Internet of Things (MIoT)

The integration of large language models (LLM) with multimodal agentic artificial intelligence (AI) within the medical Internet of Things (MIoT) ecosystem is redefining modern healthcare intelligence. This convergence enables continuous patient observation, adaptive clinical decision-making, and context-aware interaction between humans and machines across various biomedical data modalities. Healthcare systems generate a wide range of multimodal data, including textual records such as EHRs, prescriptions, and pathology notes; medical imagery such as CT, MRI, fundus, and radiographs; spoken data from consultations and transcriptions; video streams for rehabilitation and physiotherapy monitoring; and sensor readings such as ECG, SpO \({}_{2}\) , and glucose levels. Conventional unimodal algorithms fall short in interpreting this diversity, whereas LLM-augmented agentic frameworks fuse and reason over these heterogeneous sources, grounding their outputs in medical ontologies and coordinating task-specific agents to enhance real-world clinical workflows. This paper presents a comprehensive overview of multimodal agentic AI powered by LLM for MIoT-enabled healthcare systems. Introduces a six-dimensional unified taxonomy that covers multimodal input channels, fusion mechanisms, core LLM reasoning capabilities, agentic coordination models, computational deployment layers, and ethical governance frameworks. To contextualize this taxonomy, the discussion includes a Virtual Hospital case study centered on cancer that demonstrates how multimodal signals such as imaging, genomics, patient dialogues, and clinical updates integrate through intelligent agents to enable personalized diagnosis, automated documentation, home rehabilitation, and rapid intervention in emergencies. The survey also consolidates current progress on datasets, benchmarks, and evaluation protocols for AI in multimodal and agentic healthcare. The survey identifies critical research gaps, such as the lack of longitudinal multimodal datasets, standardized evaluation frameworks for multi-agent reasoning, and reliable methods to assess trustworthiness in clinical AI. Furthermore, it examines security and compliance issues such as adversarial manipulation, data leakage, and accountability across distributed agent networks, and it proposes countermeasures through federated data governance, secure MCP-oriented orchestration, and privacy-aware edge deployment strategies. By situating recent advances within the Virtual Hospital paradigm and oncology workflows, this study provides a systematic foundation for developing scalable, secure, and ethically aligned multimodal agentic systems based on LLMs, guiding the next generation of intelligent MIoT-driven healthcare ecosystems

Authors

Institutions

Publication Details

Journal
ACM Transactions on Multimedia Computing Communications and Applications
Published
2026-04-07
DOI
https://doi.org/10.1145/3805047
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

An LLM-Enabled Multimodal Agentic AI Framework for the Medical Internet of Things (MIoT)

Syed Usman Jamil, Tanveer Zia, Mubarak Alrashoud, Muhammad Ali Paracha et al.
ACM Transactions on Multimedia Computing Communications and Applications
Artificial Intelligence in Healthcare and Education
article

An LLM-Enabled Multimodal Agentic AI Framework for the Medical Internet of Things (MIoT)

Syed Usman Jamil, Tanveer Zia, Mubarak Alrashoud, Muhammad Ali Paracha, M. Arif Khan, Selwa A. F. Al-Hazzaa, M. Shamim Hossain, Min Chen, M. A. Rahman
article en

Abstract

The integration of large language models (LLM) with multimodal agentic artificial intelligence (AI) within the medical Internet of Things (MIoT) ecosystem is redefining modern healthcare intelligence. This convergence enables continuous patient observation, adaptive clinical decision-making, and context-aware interaction between humans and machines across various biomedical data modalities. Healthcare systems generate a wide range of multimodal data, including textual records such as EHRs, prescriptions, and pathology notes; medical imagery such as CT, MRI, fundus, and radiographs; spoken data from consultations and transcriptions; video streams for rehabilitation and physiotherapy monitoring; and sensor readings such as ECG, SpO \({}_{2}\) , and glucose levels. Conventional unimodal algorithms fall short in interpreting this diversity, whereas LLM-augmented agentic frameworks fuse and reason over these heterogeneous sources, grounding their outputs in medical ontologies and coordinating task-specific agents to enhance real-world clinical workflows. This paper presents a comprehensive overview of multimodal agentic AI powered by LLM for MIoT-enabled healthcare systems. Introduces a six-dimensional unified taxonomy that covers multimodal input channels, fusion mechanisms, core LLM reasoning capabilities, agentic coordination models, computational deployment layers, and ethical governance frameworks. To contextualize this taxonomy, the discussion includes a Virtual Hospital case study centered on cancer that demonstrates how multimodal signals such as imaging, genomics, patient dialogues, and clinical updates integrate through intelligent agents to enable personalized diagnosis, automated documentation, home rehabilitation, and rapid intervention in emergencies. The survey also consolidates current progress on datasets, benchmarks, and evaluation protocols for AI in multimodal and agentic healthcare. The survey identifies critical research gaps, such as the lack of longitudinal multimodal datasets, standardized evaluation frameworks for multi-agent reasoning, and reliable methods to assess trustworthiness in clinical AI. Furthermore, it examines security and compliance issues such as adversarial manipulation, data leakage, and accountability across distributed agent networks, and it proposes countermeasures through federated data governance, secure MCP-oriented orchestration, and privacy-aware edge deployment strategies. By situating recent advances within the Virtual Hospital paradigm and oncology workflows, this study provides a systematic foundation for developing scalable, secure, and ethically aligned multimodal agentic systems based on LLMs, guiding the next generation of intelligent MIoT-driven healthcare ecosystems

ACM Transactions on Multimedia Computing Communications and Applications
King Abdulaziz City for Science and Technology (SA), Charles Sturt University (AU), King Saud University (SA), Mind Australia (AU), Islamic University of Madinah (SA), South China University of Technology (CN), The University of Notre Dame Australia (AU)
Quality Education
Openalex Percentile: Top 31%
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.