Talk Is Cheap, Show Me Your Colors: Low-Budget Consistency Auditing for Aggregated LLM Services
Aggregated large language model (LLM) services expose declared model identifiers while hiding the backend configurations that fulfill client requests. We present HuePrint, a lightweight black-box framework that audits brokered endpoint consistency via structured color elicitation (#RRGGBB and R,G,B) and continuous geometric distribution comparison without model weights or token probabilities. Across 34,528 deduplicated live cloud responses spanning 56 endpoints and 12 backends (Datasets A–E and Studies S1–S4), HuePrint lowers 30-call-per-side equal error rates over single-word baselines from 4.52–5.23% to 0.94–1.00% on the development cohort and from 6.25–8.04% to 3.49–5.00% across 20 locked endpoints, reaching 0.00% at 60–150 calls/side. Regularized RGB energy halves shallow-census error vs. exact-hex divergence (5.36% vs. 10.71%), and a hybrid color metric cuts cross-provider error from 16.67% (string MMD-U and pure energy) to 5.93%. Across three-week and cross-gateway checks, score-based color auditing keeps 0.00% ranking error where strict equality tests reject up to 42.86% of genuine pairs, and RGB geometry bounds numerical perturbations. Unified Lipschitz bounds and estimator diagnostics guide calibration: self-centering lowers clean held-out false rejections from 30% to 10%, and U-statistic energy ratios eliminate the 256-draw bootstrap. Version 2.0.0 changes: Editorial and precision refinement of the English main manuscript and supplementary material (explicit baseline and hybrid-metric attributions, clarified session-level burst-detector limitations, split dense clauses for readability, and verified ICLR 2025 proceedings pagination pp. 86369–86382 for Reference [4]). All 34,528 cloud responses, 58 SHA-256-protected data/baseline artifacts, equations, tables, and empirical numbers remain bit-for-bit unchanged from v1.0.0. This Zenodo record includes the complete preprint with supplementary material (HuePrint_Preprint.pdf, 28 pages), standalone main and supplementary PDFs, and a self-contained offline replication package (HuePrint_Replication_Package.zip) containing all 12 read-only JSONL response corpora, multilingual prompt batteries, statistical analysis scripts, unit tests, and SHA-256 integrity manifests.
Authors
- 何喜冠
Institutions
- Yunnan University of Traditional Chinese Medicine (CN)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-18
- DOI
- https://doi.org/10.5281/zenodo.24340930
- Primary Topic
- Adversarial Robustness in Machine Learning
- Type
- preprint