A generalizable model for the facial recognition of sika deer with enhanced cross-domain performance

• Animal individual identification is crucial for population monitoring and intelligent breeding. The dual-channel CNN-Transformer fusion and multi-scale feature enhancement module prioritize task-relevant features, significantly improving recognition accuracy and model performance. • The detection accuracy of sika deer face is 7.8 % higher than that of baseline YOLOv11n, which is better than 13 mainstream models. And DF-YOLO extends to multi-species (pigs/cattle) with domain generalization, boosting precision farming efficiency and sustainability. • Lightweight design reduces computational costs by one-third for edge deployment. • The multi-scale feature enhancement suppressed the noise in the complex background, and the model focused more on the facial information and eliminated the background interference without using segmentation. • Non-invasive monitoring supports animal conservation and precision farming. The generalization ability across scenarios and data sets is enhanced to provide a reliable scheme for animal monitoring and management. Animal recognition technology is recognized as a fundamental tool for ecological conservation, species monitoring, and scientific research. The population management of sika deer, a first-class protected species in China, heavily depends on precise individual identification. However, challenges such as complex backgrounds, occlusions, and age-related facial variations pose significant obstacles to robust facial recognition. To address these domain-specific limitations, an enhanced YOLOv11-based architecture, DeerFace-YOLO, is proposed. The model is validated through transfer learning across developmental stages using a curated dataset of 40 individuals with diverse postures in complex environments. To assess scalability, the framework is extended to pig facial recognition and cattle individual detection tasks, demonstrating cross-species generalization capabilities. A dual-channel architecture is implemented, integrating Vision Transformer (ViT) and Convolutional Neural Networks (CNNs) to synergize local feature extraction with global contextual modeling, thereby mitigating CNN’s limited receptive fields and ViT’s computational complexity. Background interference is suppressed through adaptive attention mechanisms. For multi-scale feature optimization, a Multi-Scale Feature Enhancement (MSFE) module is designed to dynamically prioritize task-relevant features. Additionally, a Reparameterizable Lightweight Detection Head (RLDH) is developed to maintain accuracy while reducing parameter redundancy by 33 %. Experimental results demonstrate state-of-the-art performance, achieving 98.7 % recognition accuracy and a threefold reduction in computational load compared to baseline models. The framework establishes a standardized dataset for non-invasive animal identification and provides a scalable solution for ecological conservation and precision livestock management.