Wild ActionFormer: Enhancing wildlife action recognition for 11 endangered species in Wolong

The video of wild animals captured by trap cameras provides conservationists with intuitive information on animal action, holding significant potential in ethology and ecology. This study focuses on 11 endangered wild animal species videos in the Wolong Nature Reserve and develops a video self-supervised learning-based animal action recognition network—Wild ActionFormer, to achieve automated analysis of wild animal action classes. We utilize UniformerV2 as the base backbone network, integrating self-supervised learning methods to enhance feature extraction capabilities. We constructed a differential dispersion regularization loss function to maintain the alignment of self-supervised learning features and improve the network’s robustness against interference. The introduction of the Focal Loss reweighting strategy optimizes the loss for long-tail classes, mitigating the bias towards head data. Experimental results on our released LoTE-Animal open-source dataset show that the proposed action recognition network achieves a Top-1 accuracy of 95.09%, approximately 4 percentage points higher than the baseline. The LoTE-Animal dataset comprises 10k videos, including endangered wild animals from the Wolong Nature Reserve in Sichuan, China, such as the giant panda, sambar, and Sichuan snub-nosed monkey. • An accurate action recognition framework Wild ActionFormer is designed to automatically analyze the action of wild animals. • The backbone integrates self-supervised learning methods, enhancing the network’s feature extraction. • A differential dispersion regularization loss function to maintain the alignment of self-supervised learning features. • A reweighting strategy was adopted to optimize the loss for long-tail classes. • Wild ActionFormer achieves a Top-1 accuracy of 95.09% on our LoTE-Animal dataset.