Report-Supervised Vision–Language Alignment for Label-Efficient Bronchoscopic Finding Recognition

Background/Objectives: Fine-grained annotation of bronchoscopic findings is costly, whereas every bronchoscopic image is already described in the examination report. This single-centre proof-of-concept study investigated whether routine report text can serve as supervision for bronchoscopic finding recognition. Methods: BiomedCLIP was aligned with the image–caption pairs of the publicly available Bronchoscopy Examination Report Dataset (BERD; 4826 training images) using standard contrastive learning, a caption-aware objective that treats identical captions as positives, and a label-aware reference. The encoders were evaluated on the official test set, excluding three images duplicated from the training set (313 images, 180 patients), for zero-shot recognition of 23 report-derived finding categories, cross-modal retrieval, linear probing with 1–100% of the labelled training patients, and fine-tuning, using a hierarchical patient-level bootstrap and three training seeds. Results: Without training on finding labels, alignment raised zero-shot macro AUPRC from 0.184 to 0.255–0.278 and macro AUC-ROC from 0.59 to 0.77–0.78, and the gain held across ten prompt formulations. Retrieval generalised to captions never seen during training. With 1–25% of the labelled patients, linear probes on report-aligned features outperformed DINOv2, Swin-T and pretrained BiomedCLIP; with 5%, they exceeded DINOv2 and Swin-T trained with all labels. Caption-aware training showed no measurable benefit over standard contrastive training, full fine-tuning did not improve macro F1, and absolute performance remained modest (macro F1 at most 0.32). Conclusions: In this single-centre dataset, report text was an effective source of supervision for recognising report-derived finding categories with few labels; independent reference standards and external validation are needed before clinical use.

Authors

Publication Details

Journal
Diagnostics
Published
2026-11-19
DOI
https://doi.org/10.3390/diagnostics16223878
Primary Topic
Multimodal Machine Learning Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Report-Supervised Vision–Language Alignment for Label-Efficient Bronchoscopic Finding Recognition

Ilkay Sibel Kervancı
3 links
Diagnostics
Multimodal Machine Learning Applications
article

Report-Supervised Vision–Language Alignment for Label-Efficient Bronchoscopic Finding Recognition

Ilkay Sibel Kervancı
article en

Abstract

Background/Objectives: Fine-grained annotation of bronchoscopic findings is costly, whereas every bronchoscopic image is already described in the examination report. This single-centre proof-of-concept study investigated whether routine report text can serve as supervision for bronchoscopic finding recognition. Methods: BiomedCLIP was aligned with the image–caption pairs of the publicly available Bronchoscopy Examination Report Dataset (BERD; 4826 training images) using standard contrastive learning, a caption-aware objective that treats identical captions as positives, and a label-aware reference. The encoders were evaluated on the official test set, excluding three images duplicated from the training set (313 images, 180 patients), for zero-shot recognition of 23 report-derived finding categories, cross-modal retrieval, linear probing with 1–100% of the labelled training patients, and fine-tuning, using a hierarchical patient-level bootstrap and three training seeds. Results: Without training on finding labels, alignment raised zero-shot macro AUPRC from 0.184 to 0.255–0.278 and macro AUC-ROC from 0.59 to 0.77–0.78, and the gain held across ten prompt formulations. Retrieval generalised to captions never seen during training. With 1–25% of the labelled patients, linear probes on report-aligned features outperformed DINOv2, Swin-T and pretrained BiomedCLIP; with 5%, they exceeded DINOv2 and Swin-T trained with all labels. Caption-aware training showed no measurable benefit over standard contrastive training, full fine-tuning did not improve macro F1, and absolute performance remained modest (macro F1 at most 0.32). Conclusions: In this single-centre dataset, report text was an effective source of supervision for recognising report-derived finding categories with few labels; independent reference standards and external validation are needed before clinical use.

DiagnosticsVol. 16(22)
Good health and well-being
Multimodal Machine Learning Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon