Prediction of fine-grained human activities in videos using pose-based and object-based features

Human activity prediction in videos deals with anticipating the intention of human activity before it is fully observed. Activity prediction becomes more challenging when fine-grained details are to be considered. This paper presents a deep learning-based approach for predicting complex, fine-grained, and long-duration human actions in videos. Along with prediction, our approach also localises human action spatially with bounding boxes. This approach works by considering the sequential nature of the activities in the video. Each high-level activity is represented as a sequence of local actions (low-level activities). Given a partially observed video, local actions are detected and tracked first, and then these local detections are used for predicting future high-level actions. Fine-grained activity involves interactions with different objects, so we used a combination of the human pose feature and the object feature to predict fine-grained activity more accurately. We evaluated results on the publicly available MPPI cooking activity dataset.