Faithful but Wrong: Validating LLM Faithfulness Judges for Retrieval-Augmented Generation Under Corrupted Context

LLM-judged faithfulness is widely used to certify RAG answers, yet it is rarely checked against human labels, and it is unclear what it rewards when the retrieved context is wrong. We measure faithfulness, correctness, refusal, and adoption of injected falsehoods on the same answers under controlled context perturbations, and audit the faithfulness judge against two human annotators. From SQuAD v2.0 we build 75 answerable and 20 unanswerable questions, with six context conditions for answerable questions (clean, random distractors, related distractors, heavy noise, contradictory evidence, answer-absent) and three prompt mitigations. Three open-weight generators (Qwen3-8B, DeepSeek-R1-Distill-Qwen-32B, Llama-3.1-8B) produce 4,230 context-conditioned answers, scored with Ragas by a local 27B judge and rule-based checks, plus two cross-family API judges (Gemini, Claude). For Qwen3-8B, distractors had no large effect (equivalent to zero within +/-10 points under lenient matching). In all three models, a contradictory passage was the damaging perturbation: under baseline and refusal prompts, correctness fell by 20-35 points and models repeated the injected falsehood in 23-48% of answers, almost never flagging the conflict. The local judge rated 86-94% of these answers faithful. Human annotators agreed closely with each other (kappa = 0.96 on supported versus not supported). Re-weighted within each condition, the local judge agreed with humans on 91-92% of Qwen3-8B answers (kappa = 0.62-0.69), but its errors concentrated in its unfaithful verdicts. Gemini and Haiku had numerically higher agreement on the same answers, though not significantly so. A judge that mostly agrees with humans about grounding can still certify a wrong answer, so faithfulness judgments cannot stand in for correctness. The effect of prompt mitigations depends on the model.

Authors

Publication Details

Journal
Preprints.org
Published
2026-11-17
DOI
https://doi.org/10.20944/preprints202611.1682.v1
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Faithful but Wrong: Validating LLM Faithfulness Judges for Retrieval-Augmented Generation Under Corrupted Context

Kofi Nyarko, Bikesh Regmi, Md Rahman
1 links
Preprints.org
Topic Modeling
preprint

Faithful but Wrong: Validating LLM Faithfulness Judges for Retrieval-Augmented Generation Under Corrupted Context

Kofi Nyarko, Bikesh Regmi, Md Rahman
preprint en

Abstract

LLM-judged faithfulness is widely used to certify RAG answers, yet it is rarely checked against human labels, and it is unclear what it rewards when the retrieved context is wrong. We measure faithfulness, correctness, refusal, and adoption of injected falsehoods on the same answers under controlled context perturbations, and audit the faithfulness judge against two human annotators. From SQuAD v2.0 we build 75 answerable and 20 unanswerable questions, with six context conditions for answerable questions (clean, random distractors, related distractors, heavy noise, contradictory evidence, answer-absent) and three prompt mitigations. Three open-weight generators (Qwen3-8B, DeepSeek-R1-Distill-Qwen-32B, Llama-3.1-8B) produce 4,230 context-conditioned answers, scored with Ragas by a local 27B judge and rule-based checks, plus two cross-family API judges (Gemini, Claude). For Qwen3-8B, distractors had no large effect (equivalent to zero within +/-10 points under lenient matching). In all three models, a contradictory passage was the damaging perturbation: under baseline and refusal prompts, correctness fell by 20-35 points and models repeated the injected falsehood in 23-48% of answers, almost never flagging the conflict. The local judge rated 86-94% of these answers faithful. Human annotators agreed closely with each other (kappa = 0.96 on supported versus not supported). Re-weighted within each condition, the local judge agreed with humans on 91-92% of Qwen3-8B answers (kappa = 0.62-0.69), but its errors concentrated in its unfaithful verdicts. Gemini and Haiku had numerically higher agreement on the same answers, though not significantly so. A judge that mostly agrees with humans about grounding can still certify a wrong answer, so faithfulness judgments cannot stand in for correctness. The effect of prompt mitigations depends on the model.

Preprints.org
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon