Report-Supervised Vision–Language Alignment for Label-Efficient Bronchoscopic Finding Recognition
Background/Objectives: Fine-grained annotation of bronchoscopic findings is costly, whereas every bronchoscopic image is already described in the examination report. This single-centre proof-of-concept study investigated whether routine report text can serve as supervision for bronchoscopic finding recognition. Methods: BiomedCLIP was aligned with the image–caption pairs of the publicly available Bronchoscopy Examination Report Dataset (BERD; 4826 training images) using standard contrastive learning, a caption-aware objective that treats identical captions as positives, and a label-aware reference. The encoders were evaluated on the official test set, excluding three images duplicated from the training set (313 images, 180 patients), for zero-shot recognition of 23 report-derived finding categories, cross-modal retrieval, linear probing with 1–100% of the labelled training patients, and fine-tuning, using a hierarchical patient-level bootstrap and three training seeds. Results: Without training on finding labels, alignment raised zero-shot macro AUPRC from 0.184 to 0.255–0.278 and macro AUC-ROC from 0.59 to 0.77–0.78, and the gain held across ten prompt formulations. Retrieval generalised to captions never seen during training. With 1–25% of the labelled patients, linear probes on report-aligned features outperformed DINOv2, Swin-T and pretrained BiomedCLIP; with 5%, they exceeded DINOv2 and Swin-T trained with all labels. Caption-aware training showed no measurable benefit over standard contrastive training, full fine-tuning did not improve macro F1, and absolute performance remained modest (macro F1 at most 0.32). Conclusions: In this single-centre dataset, report text was an effective source of supervision for recognising report-derived finding categories with few labels; independent reference standards and external validation are needed before clinical use.
Authors
- Ilkay Sibel Kervancı (ORCID: https://orcid.org/0000-0001-5547-1860)
Publication Details
- Journal
- Diagnostics
- Published
- 2026-11-19
- DOI
- https://doi.org/10.3390/diagnostics16223878
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00