Evaluating large language model-generated oncology nutrition responses: a conceptual framework and validation roadmap
Large language models (LLMs) are increasingly accessible to patients with cancer as sources of immediate information about diet, symptoms, supplements, and treatment-related nutritional concerns. Their conversational fluency may create an illusion of clinical adequacy: a response can be well written, plausible, and apparently personalized while remaining incomplete, poorly contextualized, or clinically unsafe. This problem is particularly relevant in oncology nutrition, where advice about restrictive diets, fasting, supplements, malnutrition, sarcopenia, cachexia, and treatment tolerance may have direct implications for patient safety. Existing approaches to evaluating LLM-generated health information use heterogeneous criteria, including accuracy, completeness, readability, usefulness, personalization, evidence support, trust, safety, and potential harm. Classical instruments such as DISCERN, the Patient Education Materials Assessment Tool, and the JAMA benchmarks address selected dimensions of health information quality, whereas contemporary LLM-specific frameworks increasingly account for conversational, clinical, and safety-related features. However, these approaches are not specifically designed for oncology nutrition and do not consistently operationalize a non-compensatory logic in which clinically serious safety concerns are assessed separately from otherwise favorable response qualities. We propose a conceptual and operational framework for evaluating patient-facing LLM-generated oncology nutrition responses. The framework treats scientific, clinical, and nutritional appropriateness; clinical and nutritional safety; communication quality and patient utility; transparency; and system-level robustness as conceptually distinct and non-substitutable evaluation dimensions. It also introduces critical safety flags intended to identify veto-level concerns that should not be averaged with favorable performance in other domains. The framework is intended to guide the development of a criterion-referenced expert rubric, support structured evaluation of oncology nutrition responses, and define testable propositions for future validation across models, clinical scenarios, languages, and healthcare contexts.
Authors
- Iván Herrera‐Peco (ORCID: https://orcid.org/0000-0002-5183-5679)
- Lara Garcelán González
- Sergio Izcara (ORCID: https://orcid.org/0000-0002-7315-2713)
- María Aránzazu Sánchez-Calabuig
- Raquel Moreno-Sánchez (ORCID: https://orcid.org/0000-0002-2642-1562)
- Cristina Martínez Roldán
- Eduardo García-Rico
- Aurea Esparza de la Sota
- Carlos Ruíz Núñez
Institutions
- HM Hospitales (ES)
- Centro de Emergencias Sanitarias 061 (ES)
- Fundación de Investigación HM Hospitales (ES)
- Camilo José Cela University (ES)
Publication Details
- Journal
- Frontiers in Digital Health
- Published
- 2026-11-16
- DOI
- https://doi.org/10.3389/fdgth.2026.1929741
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Universidad Camilo José Cela