Untrained low-cost vision–language models for recognising and marking eosinophils in oesophageal biopsies: an exploratory diagnostic accuracy study

Background. Diagnosing eosinophilic oesophagitis requires counting eosinophils in biopsies. Whether general-purpose vision–language models (VLMs) can recognise and mark eosinophils without training, and at what cost, is unknown. Methods. In this exploratory diagnostic accuracy study, six low-cost VLMs were used without training on public haematoxylin and eosin whole-slide images (17 test slides, 15 patients). One gastrointestinal pathologist (the author) set the reference standard. The VLMs classified 1 200 single-cell crops (600 eosinophils) with no instruction, expert text, or text plus six labelled images; the pre-specified primary contrast was balanced accuracy with examples versus no instruction (Holm-adjusted McNemar). Secondary analyses covered marking every eosinophil in 112 fields of 128 µm (652 eosinophils; cell-level F1), harness variants, a three-VLM council checking a trained detector, and five new patients. After testing, two independent pathologists marked the same fields. 95% CIs came from a slide-cluster bootstrap. Results. Examples raised balanced accuracy in every model, by 0.048 (95% CI 0.022–0.071) to 0.303 (0.281–0.324) (Holm p < 0.001), to 0.828 (0.807–0.849) at best (EUR 0.08 per 1 000 crops); the best model was 0.061 (0.034–0.085) below the pathologist's blind re-read (post hoc). In fields the best model reached F1 0.691 (0.655–0.724) over the fields the independent pathologists marked, against 0.749 (0.710–0.779) for them (−0.059; −0.082 to −0.028; post hoc) and 0.844 for the reference pathologist's blind first pass, which helped build the reference. Classifying candidate cells as crops, the best of about 50 training-free configurations, reached 0.737 (0.699–0.777); on new patients the comparable arm reached 0.651 (0.559–0.729) and neither pre-specified contrast was detectable. The council showed no detectable gain over the detector (−0.020; −0.044 to 0.007). Conclusions. Untrained low-cost VLMs recognised eosinophils reliably only with labelled examples and marked them less accurately than pathologists; accuracy, not cost, limited them. With one reference pathologist, two independent readers and one reused cohort, these results are hypothesis-generating. Companion preprint: doi:10.5281/zenodo.24061259. Files: main text and supplementary material.