Model Choice Meets Prompt Choice : A Dual-Factor Study of Zero-Shot Low-Resource Plant Recognition

In this study, we have assessed the zero-shot classification performance of Jeju Island plant images using five multimodal vision–language models: CLIP, SigLIP, SigLIP Multilingual, SigLIP SO400M, and SigLIP2. Evaluation data comprised image–text pairs of plant species collected from four ecologically distinct regions (Deonggae Coast, Min-oleum, Jabaebong, and Jeju City). All models were evaluated under an identical zero-shot classification protocol to ensure a fair comparison. Among them, SigLIP SO400M achieved the highest accuracy on the Deonggae Coast subset, attaining a macro accuracy of 0.7460 and a micro accuracy of 0.7612, thereby outperforming the other models. The prompt language format exerted a significant influence on performance: English-only prompts consistently surpassed Korean-only prompts across all models. Confusion matrix analysis revealed region-specific class-level misclassification patterns, identifying species prone to frequent confusion. Collectively, these results demonstrate the robust zero-shot classification capabilities of contemporary vision–language models for fine-grained plant species identification and underscore the importance of selecting both an appropriate model and prompt format for a given task. The code used for these experiments is publicly available at github.com/flyaround365/JejuPlantsClassification.