IA4CACAO: Deep Learning-Based Classification of Fermented Cocoa Beans (Cut Test Images) in Colombia

Automated and objective grading of cocoa (Theobroma cacao L.) fermentation remains a major challenge because the conventional cut test relies on subjective visual inspection and is difficult to scale. In this study, we develop and evaluate a deep learning pipeline for classifying cocoa bean fermentation levels from expert-annotated cut-test images acquired under controlled conditions, enabling the systematic evaluation and comparison of multiple convolutional and transformer-based architectures under consistent preprocessing, training, and evaluation protocols. The dataset comprises 4347 segmented cocoa bean images distributed across four severely imbalanced classes, namely fermented, under-fermented, slaty, and violet. Representative architectures, including EfficientNet-B0, MobileNetV3-Large, ConvNeXt-XLarge, ViT-Base, and ViT-Large, are benchmarked to analyze the effects of class imbalance, RGB versus HSV color representation, training duration, and label-space formulation. The results show that severe class imbalance strongly degrades performance in direct four-class classification. A hierarchical binary-to-multiclass strategy significantly improves balanced recognition by separating fermented from unfermented beans prior to subclass discrimination, increasing macro-F1 scores from approximately 80–83% to 89–91%. Among the evaluated models, ViT-Base emerges as the most stable architecture across experimental settings and offers the best balance between classification performance, training stability, and computational cost. Although larger models achieve slightly higher peak performance under balanced conditions, ViT-Base provides more consistent results under realistic constraints. The proposed framework enables near-real-time inference on segmented single-bean images and supports objective, reproducible, and scalable fermentation assessment. These findings demonstrate that performance in cocoa fermentation grading is determined not only by model capacity, but also by imbalance-aware label-space design and evaluation protocols aligned with real-world cut-test conditions.