Artificial intelligence large language models and the threat to human health – A consensus statement by AAIO

Artificial intelligence large language models (LLMs) are increasingly used by patients, clinicians, and health organizations to obtain, summarize, and generate medical information. Although these tools can improve access to information and reduce communication barriers, their fluent output can obscure important limitations. This narrative review examines how LLMs may harm human health through interacting technical, cognitive, clinical, and societal mechanisms. Risks include fabricated or incomplete information, contextually inappropriate advice, failure to identify emergencies, unsafe medication or procedural guidance, sycophantic responses to leading questions, and excessive user reliance. Vulnerable people seeking mental health support may be particularly susceptible to validating or dependency-promoting interactions. Beyond individual encounters, LLMs can amplify health misinformation, contaminate evidence-retrieval systems, reproduce inequities, expose sensitive health data, and undermine scientific integrity. Case reports and recent litigation illustrate potential severity but do not establish the incidence of harm or prove causation. Risk reduction requires complementary action by patients, clinicians, developers, health systems, and regulators. LLMs should be treated as fallible communication tools rather than autonomous clinicians: High-stakes outputs require independent verification, identifiable data require appropriate protection, and urgent symptoms require conventional emergency assessment. Evaluation using realistic patient conversations, sustained human oversight, and transparent post-deployment monitoring is essential if the benefits of medical LLMs are to be realized without avoidable harm. Real-world cases have been included to provide greater clarity on the gravity of the situation and the finer points of jurisprudence under consideration.

Authors

Institutions

Publication Details

Journal
Indian Journal of Medical Sciences
Published
2026-11-21
DOI
https://doi.org/10.25259/ijms_202_2026
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Artificial intelligence large language models and the threat to human health – A consensus statement by AAIO

Nitin Yashas, Pavan Raghava Reddy Kalluri, Prathyusha Eaga, Suresh VS Attili et al.
Indian Journal of Medical Sciences
Artificial Intelligence in Healthcare and Education
article

Artificial intelligence large language models and the threat to human health – A consensus statement by AAIO

Nitin Yashas, Pavan Raghava Reddy Kalluri, Prathyusha Eaga, Suresh VS Attili, Sandeep Batra, Purvish M. Parikh
article en

Abstract

Artificial intelligence large language models (LLMs) are increasingly used by patients, clinicians, and health organizations to obtain, summarize, and generate medical information. Although these tools can improve access to information and reduce communication barriers, their fluent output can obscure important limitations. This narrative review examines how LLMs may harm human health through interacting technical, cognitive, clinical, and societal mechanisms. Risks include fabricated or incomplete information, contextually inappropriate advice, failure to identify emergencies, unsafe medication or procedural guidance, sycophantic responses to leading questions, and excessive user reliance. Vulnerable people seeking mental health support may be particularly susceptible to validating or dependency-promoting interactions. Beyond individual encounters, LLMs can amplify health misinformation, contaminate evidence-retrieval systems, reproduce inequities, expose sensitive health data, and undermine scientific integrity. Case reports and recent litigation illustrate potential severity but do not establish the incidence of harm or prove causation. Risk reduction requires complementary action by patients, clinicians, developers, health systems, and regulators. LLMs should be treated as fallible communication tools rather than autonomous clinicians: High-stakes outputs require independent verification, identifiable data require appropriate protection, and urgent symptoms require conventional emergency assessment. Evaluation using realistic patient conversations, sustained human oversight, and transparent post-deployment monitoring is essential if the benefits of medical LLMs are to be realized without avoidable harm. Real-world cases have been included to provide greater clarity on the gravity of the situation and the finer points of jurisprudence under consideration.

Indian Journal of Medical SciencesVol. 78
Guntur Medical College (IN), Max Healthcare (IN), Krishna Institute of Medical Sciences (IN), Manipal Hospital (IN)
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon