Poisson Flow Based Conditional Generative Automatic Music Transcription

The inception of musical notation is widely regarded as a key contributor to the evolution and propagation of music. It represents a shift from music being an abstract auditory experience to a form articulated through specific notations and established symbols. With mastery of this information, one can easily reconstruct sound from the score. Given the musical knowledge, time, and effort required for manual transcription, automatic music transcription (AMT) has been proposed and emerged as a crucial subtask in the domain of music information retrieval (MIR) within music and artificial intelligence fields, boasting a wealth of downstream applications. The AMT task entails extracting features from input music data and transcribing them into symbolic representations, typically piano rolls or Musical Instrument Digital Interface (MIDI) notes. Classic methods will process the AMT task as a discriminative task that heavily rely on the labels. Such models necessitate high label accuracy and an abundance of music data to deliver commendable accuracy. Despite this, there is a dearth of frameworks for unsupervised learning within the context of AMT. This paper introduces a conditional generative AMT framework based on the Poisson Flow Generative Model (PFGMPP), which reinterprets the discriminative AMT task as a conditional MIDI note generation task based on provided music data. Our framework facilitates unsupervised pre-training and transfer learning capabilities for AMT task. In our experiments, we evaluate the application of the PFGMPP model to the AMT task, explore the optimal parameter settings, investigate the impact of incorporating generative model, and conduct transfer learning for various low-resource instruments. We also carry out the basic music generation and music infilling tests on our framework. The experiment results reveal the applicability of the PFGMPP model to AMT and its capacity to perform unconditional generation. In contrast to the discriminative model, our framework attains a 13.9 points enhancement in note-wise \(F_{1}\) score. Moreover, our framework outperforms than the existing state-of-the-art (SOTA) unsupervised models by 1.6 points in note-wise \(F_{1}\) score, demonstrating its superior unsupervised pre-training power.