Do Language Models Say "I Don't Know What You Mean"? Evaluating the Reliability of LLM-Generated Explanations Under Ambiguous and Incomplete Prompts

Large Language Models (LLMs) explain things fluently, and that fluency is exactly the problem: a smooth,confident answer gives the reader very little reason to doubt it. When a question is ambiguous (it can be read inmore than one way) or incomplete (it leaves out facts needed for a specific answer), a model can quietly chooseone reading or invent the missing details, and the user may never find out. This paper reports a small, matchedpilot study of that behaviour in ChatGPT and Google Gemini. We wrote five base tasks across five domains,expressed each as a clear, an ambiguous and an incomplete prompt (15 prompts), and submitted all of them toboth models in fresh chats, giving 30 responses. Each response was rated on factual support, relevance,completeness, consistency and uncertainty handling, and was also coded for unsupported assumptions and forclarification behaviour. The ratings reported here come from a single AI-assisted evaluator and are preliminary;independent human ratings and inter-rater agreement are still pending. Within those limits, the pattern isconsistent. Reliability fell for both models once the prompt was no longer clear. On ambiguous prompts, neithermodel flagged the ambiguity in any of the ten responses, and on incomplete prompts clarification requests wererare (1 of 10 responses). We offer the prompt set, rubric and scoring formulas as a repeatable method for largerstudies.Keywords: large language models, explanation reliability, ambiguous prompts, incomplete prompts,unsupported assumptions, clarification behaviour, human evaluation

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-19
DOI
https://doi.org/10.5281/zenodo.24362649
Primary Topic
Explainable Artificial Intelligence (XAI)
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Do Language Models Say "I Don't Know What You Mean"? Evaluating the Reliability of LLM-Generated Explanations Under Ambiguous and Incomplete Prompts

Aarti Tanaji Katkar
Zenodo (CERN European Organization for Nuclear Research)
Explainable Artificial Intelligence (XAI)
preprint

Do Language Models Say "I Don't Know What You Mean"? Evaluating the Reliability of LLM-Generated Explanations Under Ambiguous and Incomplete Prompts

Aarti Tanaji Katkar
preprint en

Abstract

Large Language Models (LLMs) explain things fluently, and that fluency is exactly the problem: a smooth,confident answer gives the reader very little reason to doubt it. When a question is ambiguous (it can be read inmore than one way) or incomplete (it leaves out facts needed for a specific answer), a model can quietly chooseone reading or invent the missing details, and the user may never find out. This paper reports a small, matchedpilot study of that behaviour in ChatGPT and Google Gemini. We wrote five base tasks across five domains,expressed each as a clear, an ambiguous and an incomplete prompt (15 prompts), and submitted all of them toboth models in fresh chats, giving 30 responses. Each response was rated on factual support, relevance,completeness, consistency and uncertainty handling, and was also coded for unsupported assumptions and forclarification behaviour. The ratings reported here come from a single AI-assisted evaluator and are preliminary;independent human ratings and inter-rater agreement are still pending. Within those limits, the pattern isconsistent. Reliability fell for both models once the prompt was no longer clear. On ambiguous prompts, neithermodel flagged the ambiguity in any of the ten responses, and on incomplete prompts clarification requests wererare (1 of 10 responses). We offer the prompt set, rubric and scoring formulas as a repeatable method for largerstudies.Keywords: large language models, explanation reliability, ambiguous prompts, incomplete prompts,unsupported assumptions, clarification behaviour, human evaluation

Zenodo (CERN European Organization for Nuclear Research)
JSPM University (IN)
Explainable Artificial Intelligence (XAI)
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon