Multi-Signal Informed Attention for Beat and Downbeat Detection

Processing multiple signal sources presents a challenge but also an opportunity in music information processing, especially when the sources provide complementary information. The attention mechanism paves the way toward addressing this challenge. In this paper, a novel transformer for beat and down-beat detection is proposed, named Informed Beat Transformer. It is theorised to both improve upon previous beat and downbeat detection models and take advantage of auxiliary information. Two experiments are run to test these two hypotheses. In the first experiment, it is directly compared to the Beat Transformer and found to have an average improvement of 0.005 in the F1 score across 4 datasets. The second experiment compares the Beat Transformer and madmom's beat tracker to the Informed Beat Transformer in a situation where a beat coherent signal stream is available, in this case a drum track. It was found to have a significant improvement to the Beat Transformer with an increase in F1 score of 0.702 and an increase of 0.018 to the madmom beat tracker. These results show the efficacy of the Informed Beat Transformer in both experimental settings.