Automated Human Activity Recognition Using Attention Integrated Hybrid Conv-LSTM Model

Automated human activity recognition in videos has been of immense importance lately due to usage in the field of security and surveillance, patient monitoring, robotics, etc. Hence, this paper presents a novel hybrid model that combines the DenseNe201 convolutional neural network with Long Short Term Memory (LSTM) model and enhances it by incorporating multi-head attention mechanism for efficient Human Activity Recognition. The spatial feature is extracted using DenseNet201 architecture while the temporal relationship between the video frames is captured using LSTM layers and the features are assigned relative importance using the multi-head attention layers. The work analyzes the proposed model using standard UCF50 dataset and also trains other state-of-art models namely, Xception, MobileNetV2, InceptionV3, as well as a 3D CNN, etc with LSTM network on the dataset for comprehensive comparisons. The proposed hybrid approach has been found to deliver the best results as demonstrated by an F1-score of 80%. It also derives real-time activities from a single video input, essentially making the model practical.