PB-STR: A spatiotemporal transformer network for multi-behavior recognition of pigs

• A pig behavior recognition method based on a spatiotemporal transformer is proposed. • Multiple pig behaviors are effectively recognized within a single camera’s field of view. • UnetTSF and CAA enhance temporal continuity and spatial feature extraction. • The method achieves 94.2% mAP, 87.5% precision, and 90.8% recall in recognition. Pig behavior is a reliable indicator of health status, accurate recognition is vital for effective health surveillance and management. This study proposes PB-STR, a behavior recognition model based on the integration of video spatiotemporal feature fusion. The model addresses challenges in recognizing multiple behaviors within a single frame and handling dynamically changing behaviors. It develops a Time Series Prediction Module (UnetTSF) and a Context Anchor Attention (CAA) module, enhancing the PB-STR framework's ability to capture feature evolution over time and fully utilize contextual information. To enhance the model's proficiency in detecting and recognizing behaviors within overlapping regions, the detection head employs Minimum Points Distance Intersection over Union (MPDIoU) as its bounding box loss function, improving adaptability to variations in pig positions. The PB-STR model was evaluated on a proprietary dataset of 294 videos covering seven pig behaviors. With a mean Average Precision of 94.2%, recall of 90.8%, and precision of 87.5%, the PB-STR model can concurrently recognize five dynamic and two static behaviors in pigs. By outperforming models such as DETR, DAB-DETR, Deformable DETR, CenterNet, and DINO, the proposed approach not only enhances detection accuracy but also serves as a technological foundation for intelligent, welfare-oriented pig farming, facilitating in the sector's modernization.