Evaluating large language model-generated oncology nutrition responses: a conceptual framework and validation roadmap

Large language models (LLMs) are increasingly accessible to patients with cancer as sources of immediate information about diet, symptoms, supplements, and treatment-related nutritional concerns. Their conversational fluency may create an illusion of clinical adequacy: a response can be well written, plausible, and apparently personalized while remaining incomplete, poorly contextualized, or clinically unsafe. This problem is particularly relevant in oncology nutrition, where advice about restrictive diets, fasting, supplements, malnutrition, sarcopenia, cachexia, and treatment tolerance may have direct implications for patient safety. Existing approaches to evaluating LLM-generated health information use heterogeneous criteria, including accuracy, completeness, readability, usefulness, personalization, evidence support, trust, safety, and potential harm. Classical instruments such as DISCERN, the Patient Education Materials Assessment Tool, and the JAMA benchmarks address selected dimensions of health information quality, whereas contemporary LLM-specific frameworks increasingly account for conversational, clinical, and safety-related features. However, these approaches are not specifically designed for oncology nutrition and do not consistently operationalize a non-compensatory logic in which clinically serious safety concerns are assessed separately from otherwise favorable response qualities. We propose a conceptual and operational framework for evaluating patient-facing LLM-generated oncology nutrition responses. The framework treats scientific, clinical, and nutritional appropriateness; clinical and nutritional safety; communication quality and patient utility; transparency; and system-level robustness as conceptually distinct and non-substitutable evaluation dimensions. It also introduces critical safety flags intended to identify veto-level concerns that should not be averaged with favorable performance in other domains. The framework is intended to guide the development of a criterion-referenced expert rubric, support structured evaluation of oncology nutrition responses, and define testable propositions for future validation across models, clinical scenarios, languages, and healthcare contexts.

Authors

Institutions

Publication Details

Journal
Frontiers in Digital Health
Published
2026-11-16
DOI
https://doi.org/10.3389/fdgth.2026.1929741
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Evaluating large language model-generated oncology nutrition responses: a conceptual framework and validation roadmap

Iván Herrera‐Peco, Lara Garcelán González, Sergio Izcara, María Aránzazu Sánchez-Calabuig et al.
6 links
Frontiers in Digital Health
Artificial Intelligence in Healthcare and Education
article

Evaluating large language model-generated oncology nutrition responses: a conceptual framework and validation roadmap

Iván Herrera‐Peco, Lara Garcelán González, Sergio Izcara, María Aránzazu Sánchez-Calabuig, Raquel Moreno-Sánchez, Cristina Martínez Roldán, Eduardo García-Rico, Aurea Esparza de la Sota, Carlos Ruíz Núñez
article en

Abstract

Large language models (LLMs) are increasingly accessible to patients with cancer as sources of immediate information about diet, symptoms, supplements, and treatment-related nutritional concerns. Their conversational fluency may create an illusion of clinical adequacy: a response can be well written, plausible, and apparently personalized while remaining incomplete, poorly contextualized, or clinically unsafe. This problem is particularly relevant in oncology nutrition, where advice about restrictive diets, fasting, supplements, malnutrition, sarcopenia, cachexia, and treatment tolerance may have direct implications for patient safety. Existing approaches to evaluating LLM-generated health information use heterogeneous criteria, including accuracy, completeness, readability, usefulness, personalization, evidence support, trust, safety, and potential harm. Classical instruments such as DISCERN, the Patient Education Materials Assessment Tool, and the JAMA benchmarks address selected dimensions of health information quality, whereas contemporary LLM-specific frameworks increasingly account for conversational, clinical, and safety-related features. However, these approaches are not specifically designed for oncology nutrition and do not consistently operationalize a non-compensatory logic in which clinically serious safety concerns are assessed separately from otherwise favorable response qualities. We propose a conceptual and operational framework for evaluating patient-facing LLM-generated oncology nutrition responses. The framework treats scientific, clinical, and nutritional appropriateness; clinical and nutritional safety; communication quality and patient utility; transparency; and system-level robustness as conceptually distinct and non-substitutable evaluation dimensions. It also introduces critical safety flags intended to identify veto-level concerns that should not be averaged with favorable performance in other domains. The framework is intended to guide the development of a criterion-referenced expert rubric, support structured evaluation of oncology nutrition responses, and define testable propositions for future validation across models, clinical scenarios, languages, and healthcare contexts.

Frontiers in Digital HealthVol. 8
HM Hospitales (ES), Centro de Emergencias Sanitarias 061 (ES), Fundación de Investigación HM Hospitales (ES), Camilo José Cela University (ES)
Universidad Camilo José Cela
Good health and well-being
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon