A deep reinforcement learning approach to dance movement analysis

Although the field of deep learning has improved video-based dance classification, traditional models are computationally inefficient because they process redundant frames, and do not have the capability to center on discriminative key moments. In order to fill this gap, the present paper proposes the Reinforcement-based Attentive Temporal Sampling (RATS) framework. RATS proposes a new algorithmic framework in that it develops the classification as a sequential decision-making process. It is a three-part modular structure, which includes a custom feature extraction pipeline (based on 3D Convolutional Neural Networks (3DCNN)) to learn rich visual-motion representations, a Deep Q-Network (DQN) agent with Bidirectional Long Short-Term Memory (BiLSTM) memory to learn the optimal movement policy in the video, and a final classification head that predicts the dance style based on the state summary presented by the BiLSTM. This approach, while significantly reducing the computational complexity and focusing on important frames, significantly improves the accuracy of the model in recognizing complex dance styles; in such a way that in the evaluation on the Let's Dance dataset, it achieved the accuracy of 92.1%, showing a 3.9% increase in accuracy and a 4% improvement in the F-measure, which indicates the outstanding efficiency and effectiveness of RATS.