On the Lack of a Perceptually Motivated Evaluation Metric for Packet Loss Concealment in Networked Music Performances

Packet loss concealment (PLC) is vital in preserving audio quality for networked music performances. Although existing PLC techniques primarily target speech transmission, the unique challenges in music signals, such as complex harmonic structures and diverse timbral ranges, have yet to be adequately addressed. This is in part a result of the fact that a satisfactory objective evaluation metric for music PLC methods is missing. As a first foundational step toward this direction, this paper proposes a novel evaluation metric that leverages insights from music psychoacoustics and uses the constant-Q transform to better quantify glitch audibility induced by unconcealed packet loss (i.e., replaced with zeros) compared with existing metrics. The authors conducted extensive subjective listening tests leading to the creation of a publicly available ground truth data set, mapping objective audio features to human assessments of glitch audibility. Results show that the developed metric outperforms other measures (such as mean squared error and mean absolute error) in predicting perceptual impacts, taking a step toward addressing the need for a specialized metric for PLC in the domain of networked music performances. However, further improvements are needed to match human perceptual accuracy, which calls for further research on the development of a reliable perceptually motivated evaluation metric.