When Language Models Face Questions They Cannot Reliably Answer: Do They Still Prefer Confident Guessing Over Honest Abstention?

Large language models often produce confident falsehoods on obscure questions. Recent work argues that this is partly driven by training and evaluation incentives that reward guessing over admitting ignorance. We probed four consumer chat models (ChatGPT, Gemini Pro, Grok, and Claude) on 60 high-uncertainty AI-history questions (n = 240 model-item pairs per prompt condition) under three prompts: a standard prompt that demands a best answer (A), a prompt that permits "I don't know" (B), and a prompt that states a +1/0/-1 scoring rule (C). In this pilot, observed hallucinations fell from 11/240 (4.6%) under A to 2/240 (0.8%) under both B and C (paired exact McNemar test, p = 0.012 for each comparison; Fisher's exact test, p = 0.021). Abstention rose from 12/240 (5.0%) to 40/240 (16.7%) under B and 38/240 (15.8%) under C. The gain in reliability came at the cost of coverage: of the 28 answers that became abstentions under B, 19 had been correct. We found no detectable difference between B and C, so stating an explicit scoring rule added nothing beyond permission to abstain. The abstention shift was significant for Gemini Pro and Grok but not for ChatGPT or Claude individually, and hallucinations were rare in every condition. In a flip test in which a correct answer was challenged with "Another AI model got this wrong", all 22 eligible challenged answers were kept. The sample is small, graded against a single answer key and run through consumer interfaces, so we treat the results as a pilot. Research conducted at the AI x Epistemics Research Sprint, November 2026.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-21
DOI
https://doi.org/10.5281/zenodo.24425144
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

When Language Models Face Questions They Cannot Reliably Answer: Do They Still Prefer Confident Guessing Over Honest Abstention?

Stephen Obiajulu Ezennwa
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
article

When Language Models Face Questions They Cannot Reliably Answer: Do They Still Prefer Confident Guessing Over Honest Abstention?

Stephen Obiajulu Ezennwa
article en

Abstract

Large language models often produce confident falsehoods on obscure questions. Recent work argues that this is partly driven by training and evaluation incentives that reward guessing over admitting ignorance. We probed four consumer chat models (ChatGPT, Gemini Pro, Grok, and Claude) on 60 high-uncertainty AI-history questions (n = 240 model-item pairs per prompt condition) under three prompts: a standard prompt that demands a best answer (A), a prompt that permits "I don't know" (B), and a prompt that states a +1/0/-1 scoring rule (C). In this pilot, observed hallucinations fell from 11/240 (4.6%) under A to 2/240 (0.8%) under both B and C (paired exact McNemar test, p = 0.012 for each comparison; Fisher's exact test, p = 0.021). Abstention rose from 12/240 (5.0%) to 40/240 (16.7%) under B and 38/240 (15.8%) under C. The gain in reliability came at the cost of coverage: of the 28 answers that became abstentions under B, 19 had been correct. We found no detectable difference between B and C, so stating an explicit scoring rule added nothing beyond permission to abstain. The abstention shift was significant for Gemini Pro and Grok but not for ChatGPT or Claude individually, and hallucinations were rare in every condition. In a flip test in which a correct answer was challenged with "Another AI model got this wrong", all 22 eligible challenged answers were kept. The sample is small, graded against a single answer key and run through consumer interfaces, so we treat the results as a pilot. Research conducted at the AI x Epistemics Research Sprint, November 2026.

Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
When Language Models Face Questions They Cannot Reliably Answer: Do They Still Prefer Confident Guessing Over Honest Abstention? — Stephen Obiajulu Ezennwa · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS