Piano teaching-assisted beat recognition based on spatio-temporal two-branch attention

The challenge of beat identification in piano teaching has progressively taken the stage as intelligent education technology develops quickly. Most of the conventional beat detection systems ignore the link between the video information and the player's motions by depending just on the analysis of auditory data. Based on the spatio-temporal two-branch attention mechanism, this work presents a piano beat detection model called TempoNet to increase the accuracy and robustness of beat identification. By means of the spatio-temporal dual-branching attention mechanism, the model efficiently captures the temporal features in the audio signal and the dynamic spatio-temporal features in the video signal by deep fusion of the two-modal information. Comparatively to conventional approaches, the suggested TempoNet model shows better beat identification accuracy and robustness according to experimental results on several test datasets.