Reliability-blind tool use: How vision–language models weight stated and observed tool reliability on chest X-rays
Clinical AI pipelines increasingly place the output of a specialist classifier in a vision–language model's (VLM's) prompt and leave the VLM to weigh it. When sensitivity and specificity are equal and the tool provides independent evidence, a Bayesian observer gives substantial weight to a 99%-accurate tool and zero weight to a 50%-accurate tool. We test whether medical VLMs do this by reading the log- odds of their complete answers about chest X-rays under counterfactual tool advice and comparing the shift the advice causes with that of an ideal Bayesian observer. A discovery study on two Qwen3-VL models found deference almost insensitive to the tool's stated sensitivity and specificity. We then froze the hypotheses, measurement gates and analysis code and ran a confirmatory panel of seven open VLMs from three families (2B to 32B parameters, including a general/medical Gemma 3 pair) on 1,200 ChestX- Det and 1,994 OpenI images, 4,567,976 scored prompts in all. With reliability given as structured fields, the slope of the advice shift on the Bayes weight was 0.02–0.40 (ideal 1); every confirmatory model's upper 95% bound was below the pre-registered 0.5, and 34 of 35 model–format slopes stayed below it. All seven models followed a tool stated to be correct half the time. Plain-language sensitivity, plain accuracy, natural frequencies and an explicit instruction to weigh the tool by its reliability did not restore sensitivity (pooled slopes 0.10–0.21). The models were as reliability-blind without the image as with it, failed text-only questions about uninformative tools, ignored asymmetric reliability, and the medically tuned model was less sensitive than its general-purpose counterpart (MedGemma minus Gemma −0.16). In the three models tested, showing the tool's track record as ten labelled X-rays instead of describing it raised slopes only marginally (experienced 0.09–0.33, described −0.05 to 0.26; neither arm reached 0.5 in any model), and the order of the same ten examples moved deference more than the tool's accuracy did. Against exact-Bayes synthetic tools, blindness cost a median 12.7 balanced-accuracy points with a coin-flip tool and up to 45 points by under-using a 99%-accurate one. For the models and tasks tested, tool reliability is better applied outside the prompt, by an explicit fitted rule.
Authors
- S. M. Abdulla Al Mamun
Institutions
- University of Manchester (GB)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-19
- DOI
- https://doi.org/10.5281/zenodo.24369595
- Primary Topic
- Artificial Intelligence in Healthcare and Education
- Type
- preprint