Interpretable Deep Learning for Botanical Traits: A Comparative Study on the Role of Segmentation in Herbarium Image Analysis

The large-scale digitization of herbarium specimens—over 10 million in France via the Recolnat portal led by MNHN, and hundreds of millions worldwide—offers unprecedented opportunities for advancing plant biodiversity research. However, extracting reliable morphological traits from these images remains challenging due to the presence of non-vegetal elements (e.g., labels, rulers, envelopes) and visual background noise that can mislead model predictions. In this work, we introduce a comparative study of three deep learning models—YOLOv8, ViT, and ResNet101—applied to botanical trait classification from digitized herbarium images. We evaluate these models on both raw images and segmented versions where non-plant elements are removed. Although models trained on raw images can sometimes yield high accuracy, interpretability analysis (e.g., Grad-CAM) reveal that their predictions often rely on irrelevant background features. In contrast, segmentation consistently drives the models to focus on the plant itself, leading not only to improved performance for several morphological traits, but also laying the groundwork for explainable and scientifically meaningful AI. We argue that segmentation is not merely a preprocessing step but a prerequisite for trustworthy trait recognition and a necessary condition for enabling AI-driven discovery in botanical sciences. This work establishes a methodological foundation for interpretable plant-focused deep learning applied to biodiversity research.