Dynamic Temporal Modeling Using Cascaded Deep Networks and Encoder Transformers for Human Action Recognition

Human action recognition is an important area of research in computer vision with its vast applications from healthcare to industrial environments, solving real-world problems like smart manufacturing, video surveillance, and artificial intelligence. The increased trends of human robot interactions in workplaces and everyday life further signify this area. Gesture-based control that helps a robot recognize motions, gestures, and other physical actions allows them to assist and operate seamlessly in various interactive settings by recognizing and responding to human actions in real-time. The human action recognition models require a very high computational power to process the video frames which usually comes with a lot of cost. This paper proposes a dynamic temporal modelling for effective spatio-temporal human action recognition by simultaneously capturing the short-term, long-term, and global contextual dependencies with a balanced trade-off between accuracy and computational efficiency. Convolutional Neural Networks (CNNs) extract the spatial features, Gated Recurrent Units (GRUs) efficiently handle the short-term patterns, Long Short-Term Memory (LSTMs) support the long-term temporal dependencies and transformers enhance global understanding. The cascaded architecture integrates EfficientNetB0 with LSTMs, GRUs and transformers to achieve a superior performance with computational efficiency. The proposed model outperforms benchmark dataset UCF50 achieving an accuracy of 92% and demonstrating improved inference time and robustness.