Benchmarking Vision–Language and Vision Foundation Models for Few-Shot Defect Screening in PEM Fuel-Cell SEM Micrographs

Few-shot adaptation of vision–language models (VLMs) could reduce the labeling effort for defect screening in scanning electron microscopy (SEM) of polymer electrolyte membrane (PEM) fuel-cell materials. We built a leakage-controlled benchmark from the PEMDefect-1107 corpus to measure what VLM language priors contribute. Its controls (size-matched crops, grouping of augmented copies, validation-only selection, common calibration, 20 support draws, image-level bootstrap) were needed: with naive crops, five low-level statistics separated the classes with an area under the receiver operating characteristic curve (AUROC) of 0.951. Zero-shot prompting reached at most 0.685 Macro-F1. With 16 labeled patches per class, text-initialized adaptation of CLIP ViT-L/14 reached 0.823 and, relative to a linear probe on the same features, reduced the labels needed to reach 95% of a fine-tuned ResNet-50 from 512 to 32 per class. A linear probe on ImageNet ResNet-50 features did not differ significantly from it at any tested label budget (difference at 16 shots −0.005; 95% confidence interval −0.059 to 0.044), and two microscopy-pretrained encoders were weaker. On whole micrographs, the better 16-shot classifiers placed 55–63% of the defects in the top 10% of the area. Within each VLM, the text prototypes raised the mean AUROC but gave no detectable advantage over a strong generic encoder.