Reproducible but Incoherent: A Reference Standard-Free Reliability Benchmark of LLM-Assisted Nutritional Estimation Across 20 Model Configurations

Nutritional estimation pipelines built on large language models are entering biomarker research, but free-text meal descriptions have no ground truth, so outputs cannot be validated against a reference standard. What can be established without one is whether the same input returns the same number. We submitted 191 de-identified Spanish meal descriptions, byte-identically, three times each, at temperature 0, to 20 model configurations across two inference platforms (12,119 calls), and measured test–retest reliability by ICC(1,1), cross-model agreement, internal Atwater coherence between reported energy and macronutrients, and the reliability gradient across input specificity. All 20 models exceeded the audited cascade’s energy ICC under both comparators, but reproducibility did not imply internal coherence: the most reproducible model (ICC 1.0000) departed from its own macronutrients by more than 10% under the simplified 4/4/9 Atwater approximation in 65.58% of responses, against 3.80% for the cascade. The two axes are close to orthogonal, and the dissociation needs no threshold to state: that configuration ranks first of 20 in reliability and first of 20 in incoherence at every bound between 5% and 20%. A binary acceptance gate is not robust to that bound—3 to 17 of 20 configurations pass across the range—whereas the ordering is (Kendall tau ≥ 0.74 between any two), so coherence is reported as a ranked screening dimension with a decision curve, not as a validated criterion. Test–retest reliability is therefore necessary but not sufficient to authorise an LLM-assisted nutritional pipeline, and acceptance requires at least one reference standard-free coherence check alongside the reliability estimate.