Musical field visualization using XR technology

Sound field visualization is effective for intuitive evaluation of the field. To visualize the sound field, various methods and systems have been studied and proposed such as acoustic holography, optical methods, and sound intensity visualization. Especially, the musical field visualization is effective for evaluating musical performance, supporting hearing-impaired persons, and inspiring artistic expression. In this study, we use the sound field visualization system “OTOMIRU®.” The system consists of a 16ch microphone array, measurement system on PC, and visualization system on XR (extended reality) devices. The measurement system analyses sound sources recorded from the microphone array using the MVDR (minimum variance distortionless response) beamforming method. The analysis results are sent to the visualization system by TCP (transmission control protocol). Finally, the results are visualized as colormap-like thermography and superimposed on the real world on XR devices. The system is mainly used for noise visualization, detection, and countermeasure of construction sites or indoor spaces. To visualize the musical field, we applicate the system to visualize not only sound pressure levels but also other acoustical parameters such as frequencies. To visualize these parameters, sound field information can be detailed and comprehensible. We show the efficiency of the visualization and its applicability to other situations.