Measuring the Hermeneutic Circle in Explainable AI: A Human-Free Test Based on Data Horizons, Executable Claims, Part–Whole Agreement, and Machine Readers

Explanations of machine learning models are read by people but rarely evaluated with them, and automatic metrics score attribution vectors rather than what a reader receives. We operationalise the horizon, the hermeneutic circle, and the hermeneutics of suspicion as the Hermeneutic Circle Test (HCT), a human-free screen for explanations of tabular classifiers. HCT checks whether an explanation’s evidence lies inside a calibrated data horizon, whether its sentences survive on-horizon interventions, whether its account of a case agrees with the whole model and at which scale, and whether it helps machine readers learn the model. Over seven explainers, four datasets, and two model families (mean ± standard deviation over eight settings), the best explainer, KernelSHAP, verified 0.65 ± 0.15 of its claims, while random attributions verified 0.22 ± 0.08. On the same explanations, established faithfulness, remove-and-retrain, and disagreement metrics missed impossible evidence and narratives contradicting their own attributions, and sign-blind metrics scored sign-flipped explanations like the originals. Claim tests withstood scaffold attacks that adapt to the horizon and to the test itself; a scaffold that memorises the audited records weakened them and deceived every explainer, but part–whole agreement still flagged it. HCT is a reproducible screen to apply before user studies.

Authors

Institutions

Publication Details

Journal
Machine Learning and Knowledge Extraction
Published
2026-11-16
DOI
https://doi.org/10.3390/make8110381
Primary Topic
Explainable Artificial Intelligence (XAI)
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Measuring the Hermeneutic Circle in Explainable AI: A Human-Free Test Based on Data Horizons, Executable Claims, Part–Whole Agreement, and Machine Readers

Şadi Evren Şeker
Machine Learning and Knowledge Extraction
Explainable Artificial Intelligence (XAI)
article

Measuring the Hermeneutic Circle in Explainable AI: A Human-Free Test Based on Data Horizons, Executable Claims, Part–Whole Agreement, and Machine Readers

Şadi Evren Şeker
article en

Abstract

Explanations of machine learning models are read by people but rarely evaluated with them, and automatic metrics score attribution vectors rather than what a reader receives. We operationalise the horizon, the hermeneutic circle, and the hermeneutics of suspicion as the Hermeneutic Circle Test (HCT), a human-free screen for explanations of tabular classifiers. HCT checks whether an explanation’s evidence lies inside a calibrated data horizon, whether its sentences survive on-horizon interventions, whether its account of a case agrees with the whole model and at which scale, and whether it helps machine readers learn the model. Over seven explainers, four datasets, and two model families (mean ± standard deviation over eight settings), the best explainer, KernelSHAP, verified 0.65 ± 0.15 of its claims, while random attributions verified 0.22 ± 0.08. On the same explanations, established faithfulness, remove-and-retrain, and disagreement metrics missed impossible evidence and narratives contradicting their own attributions, and sign-blind metrics scored sign-flipped explanations like the originals. Claim tests withstood scaffold attacks that adapt to the horizon and to the test itself; a scaffold that memorises the audited records weakened them and deceived every explainer, but part–whole agreement still flagged it. HCT is a reproducible screen to apply before user studies.

Machine Learning and Knowledge ExtractionVol. 8(11)
The University of Texas at Dallas (US), Haliç University (TR)
Explainable Artificial Intelligence (XAI)
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon