P-YOWO: attention-based spatiotemporal action detection method

The task of spatiotemporal action detection plays a pivotal role in various domains such as video surveillance, medical fields, and sports analytics, necessitating real-time accuracy. One of the key challenges in spatiotemporal action detection lies in extracting temporal features. While many methods employ an additional branch for temporal feature extraction, effectively integrating the information obtained from these two branches remains a significant challenge. In this article, we propose a new framework for real-time action detection. P-YOWO, which conducts spatial and spatiotemporal modeling through two distinct branches to achieve action classification and bounding box prediction. In this endeavor, we improve the spatial modeling ability of the network for the current frame by improving the 2DCNN branch of the YOWO network structure. However, YOWO falls short in effectively leveraging spatial feature information. To address this limitation, we introduce a spatial attention mechanism in the fusion of branch information. This mechanism applies spatial self-attention to bolster the integration of spatial and temporal features in the spatial dimension. Through experiments conducted on the UCF101-24 dataset, our model exhibits an improvement of 4.8% in both frame-level and video-level average precision compared to the original model. The experimental results validate that our enhanced approach effectively explores spatial semantic information in feature maps, thereby augmenting the detection performance of the model.