A Novel Human Action Recognition Model by Grad-CAM Visualization with Multi-level Feature Extraction Using Global Average Pooling with Sequence Modeling by Bidirectional Gated Recurrent Units

Abstract Human action recognition is essential in many real-world scenarios, such as video surveillance, human–computer interaction, and behavior analysis. Despite the progress in deep learning, issues such as occlusion, distraction from the background, and motion pattern variability still exist, thus restricting the generalization ability of current models. Most methods are based only on spatial or temporal features and cannot efficiently capture both in one framework, causing lower accuracy in realistic situations. In response to these shortcomings, a multilevel feature extraction approach was proposed by integrating spatial and temporal features to improve the action recognition precision. The method captures RGB frames, optical flow, spatial saliency maps, and temporal saliency maps to enable an overall inspection of video streams. Efficient feature extraction was achieved by applying a pre-trained Inception V3 model and then bidirectional gated recurrent units (Bi-GRUs) to include sequential modeling. An attention mechanism was also included to boost the classification process by focusing on key temporal segments. UCF101 and HMDB51 benchmark datasets evaluated the efficiency of the strategy. The model’s accuracy was 98.13% on UCF101 and 81.45% on HMDB51, which validated the superior discrimination ability of the model in processing heterogeneous human actions. These results confirm that the provided framework is an efficient and discriminative action recognition approach, thus suitable for applications requiring extensive motion analysis and real-time deployment.