Issues and Challenges of Audio Technologies for the Musical Metaverse

Among all the activities envisioned for the metaverse, music has thus far received comparatively less attention. While virtual concerts and music festivals have been successful in drawing substantial audiences and increasing public attention to the idea of the metaverse, the metaverse is not ready for musicians who decide to take advantage of the distinctive features of socially immersive environments to express themselves and create music together. In this article, the authors analyze the state-of-the-art audio technologies used for the creation of shared, Audio-First immersive environments such as the musical metaverse. This work reveals important issues in consumer electronics that currently prevent the realization of a metaverse compatible with musical activities. These include hardware and software limitations used to create and experience shared immersive environments through real-time audio. This work also emphasizes two key challenges: reducing delays in network and audio processing, and addressing the lack of universal standards for spatial audio systems across different platforms. The authors believe that looking at the metaverse from the point of view of musical technologies will provide practitioners in academia and industry with key insights into what is needed to achieve true real-time activities and support human expression in the metaverse in general.