Improved ActionCLIP-based Temporal Action Recognition for Long Video

Multimodal pre-training models have been successfully applied to the vision domain, and a large number of researches have been conducted on this basis for video action recognition of short duration single actions. However, the current downstream tasks commonly have requirements for multiple action timing recognition, and there is no effective architecture to realize the requirements in terms of timing rules. This study proposes SeqActionMatchNet, a simple and effective architecture based on ActionCLIP improvement to address these challenges. The learning capability of ActionCLIP is utilized to obtain the action timing information of long videos by performing action recognition after cutting them. A generative large model is used to process the text of the task rule to obtain the text timing information, and finally the information of the video and the text are compared and matched to realize the recognition of the action timing of the long video.