A Survey of Text-to-Music Generation with Deep Learning

The field of text-to-music generation has witnessed remarkable progress in recent years, fueled by advancements in deep learning techniques. This survey paper provides a comprehensive overview of the state-of-the-art in this domain, focusing on three key aspects: model architectures, music representations, and training strategies. We first delve into the prevalent architectures employed for music generation, including language models, diffusion models, and hybrid approaches, analyzing their strengths and limitations. Subsequently, we explore various music representations, ranging from quantized waveforms and spectral features to latent representations, discussing their impact on the quality and expressiveness of the generated music. Finally, we examine different training strategies, such as standard diffusion, rectified flow, and knowledge distillation, highlighting their effectiveness and efficiency in optimizing model performance. Through this in-depth analysis, we aim to provide researchers and practitioners with a clear understanding of the current landscape and future directions in text-to-music generation. We also discuss open challenges and opportunities for further research, paving the way for the development of more sophisticated and versatile systems capable of generating high-fidelity and expressive music from natural language descriptions.