Recognizing and localizing chicken behaviors in videos based on spatiotemporal feature learning

Timely acquisition of chicken behavioral information is crucial for assessing chicken health status and production performance. Video-based behavior recognition has emerged as a primary technique for obtaining such information due to its accuracy and robustness. Video-based models generally predict a single behavior from a single video segment of a fixed duration. However, during periods of high activity in poultry, behavior transition may occur within a video segment, and existing models often fail to capture such transitions effectively. This limitation highlights the insufficient temporal resolution of video-based behavior recognition models. This study presents a chicken behavior recognition and localization model, CBLFormer, which is based on spatiotemporal feature learning. The model was designed to recognize behaviors that occur before and after transitions in video segments and to localize the corresponding time interval for each behavior. An improved transformer block, the cascade encoder-decoder network (CEDNet), a transformer-based head, and weighted distance intersection over union (WDIoU) loss were integrated into CBLFormer to enhance the model's ability to distinguish between different behavior categories and locate behavior boundaries. For the training and testing of CBLFormer, a dataset was created by collecting videos from 320 chickens across different ages and rearing densities. The results showed that CBLFormer achieved a [email protected]:0.95 of 98.34 % on the test set. The integration of CEDNet contributed the most to the performance improvement of CBLFormer. The visualization results confirmed that the model effectively captured the behavioral boundaries of chickens and correctly recognized behavior categories. The transfer learning results demonstrated that the model is applicable to chicken behavior recognition and localization tasks in real-world poultry farms. The proposed method handles cases where poultry behavior transitions occur within the video segment and improves the temporal resolution of video-based behavior recognition models. • Proposing a novel chicken behavior recognition and localization model. • The model improves temporal resolution in video-based poultry behavior monitoring. • The rationality of the model's decision-making was demonstrated by visualization. • The performance of the model was validated in real farming environments.