Continuous and Overall Evaluation of Spatial Audio Reproduction Systems With Spatially Dynamic Content

This study addresses potential challenges in evaluating reproduction systems with complex, spatially dynamic audio material because current standardized methods may lead to biased or unreliable results. To investigate this, a listening experiment was conducted comparing two assessment methods—continuous and overall evaluation—applied to two attributes, basic audio quality and surrounding, using spatially dynamic content reproduced on two reproduction systems (a stereo and a 3D surround configuration). To enable comparison with overall evaluations, continuous ratings were summarized using different strategies. Although ratings were mainly influenced by the reproduction system, spatial variation, and program item, the choice of evaluation method also had a significant effect on the scores. This effect varied depending on the attribute and the metric used to summarize the continuous data. In cases where differences were observed, continuous evaluations consistently produced higher scores than overall ratings, regardless of the summary metric. These findings indicate that continuous evaluation can capture perceptual variations over time that are lost in overall ratings, suggesting it can be a useful approach when assessing attributes influenced by spatially dynamic changes.