Reconstructing the Sound at the Edge: A Low-Latency Framework for NMP with Autotuned Local Synthesis

Networked Music Performance (NMP) systems aim to enable real-time collaboration between geographically distributed musicians. Traditional approaches transmit raw audio over IP networks, suffering from delay, jitter, and packet loss, which negatively impact the quality of experience (QoE). In this paper, we propose a novel architecture for NMP that departs from audio streaming: instead of transmitting the raw audio, we transmit symbolic musical control data (e.g., MIDI or pitch with dynamics), and synthesize the audio at the destination with potential benefits in terms of latency, bandwidth, and robustness. In addition, the new framework goes in the direction of a cloud continuum NMP, allowing the use of edge processing, such as the application of real-time effects like autotune, to improve the subjective experience. In this work, we provide the open-source code of the proposed system, and we evaluate the Quality of Experience (QoE) to assess the feasibility of the system. The proposed architecture provides a solid foundation for low-latency and scalable NMP systems with potential real-world deployment.