Deep learning under severe class imbalance: A comparative study of CNNs and YOLO models for canine white blood cell classification

• First systematic study comparing imbalance-mitigation strategies for canine white blood cell (WBC) classification across CNNs and YOLO detector-based models. • Severe natural class imbalance in the canine dataset (≈1:200 basophil-to-neutrophil ratio) exposes substantial limitations of conventional CNN approaches, even with data augmentation, class weighting, and focal loss. • YOLO detector-based architectures (YOLOv8–YOLOv12) achieve markedly higher minority-class sensitivity, improving basophil and monocyte F1-scores up to 0.85 and 0.92, respectively. • Controlled imbalance experiments (1:1, 1:25, 1:200) reveal that moderate imbalance (1:25) produces the highest performance variance for CNNs, while YOLO models remain stable. • Demonstrates that instance-centric, multi-scale representation provides a stronger solution to class imbalance than loss-level reweighting alone. • A curated, expert-annotated canine WBC dataset is provided with reproducible protocols to support future research in veterinary hematology and AI-based cytology. Automated white blood cell classification in veterinary medicine faces a significant challenge due to the extreme natural class imbalance in canine physiology. While deep convolutional neural networks (CNNs) have revolutionized human hematology, they often fail to recognize rare, clinically critical subtypes like basophils when trained on long-tail veterinary distributions. This study proposes shifting the fundamental inductive bias from traditional global CNNs to spatially aware, instance-centric YOLO detectors to provide a more robust solution than standard data-level or loss-level engineering. We introduce the KU_K9-WBC dataset, a newly curated resource of 10,262 expertly annotated single-cell images extracted from canine peripheral blood smear images, featuring a primary imbalance ratio of 1:223 (0.29% basophils vs. 65.34% neutrophils). We systematically benchmarked traditional classifiers (ResNet50 and MobileNetV2) against modern one-stage detectors (YOLOv8, v11, and v12) under varying imbalance conditions (1:1 to 1:200). Our results demonstrate that the YOLO family’s multi-scale feature fusion (FPN/PANet) and gradient-splitting backbones significantly outperform CNNs in minority-class classification. Specifically, YOLOv8 achieved a Basophil F1-score of 0.85, a substantial gain over CNNs even when the latter utilized focal loss and class-weighting. Furthermore, efficiency analysis revealed that YOLO architectures offer a 5.53-fold increase in inference speed while maintaining a lower parameter footprint, facilitating deployment on resource-constrained clinical hardware. Cross-domain validation on the human Raabin-WBC dataset confirmed that these architectural advantages remain robust across different biological morphologies and data volumes.