A Quality of Experience and Visual Attention Evaluation for 360° videos with non-spatial and spatial audio

This research presents the results of an empirical study that aimed to investigate the influence of various types of audio (spatial and nonspatial) on the quality of experience (QoE) of and visual attention in 360° videos. The study compared the eye gaze, head pose, pupil dilations, and heart rate of 73 users who watched ten 360° videos with different sound configurations. The configurations evaluated were: no sound; non-spatial (stereo) audio; and two spatial sound conditions (first and third-order ambisonics). The videos covered various categories and presented both indoor and outdoor scenarios. An analysis of the collected data was performed using various data visualization and statistical techniques. The results of the study show that users have different viewing patterns and physiological responses when watching 360° videos with different sound conditions. Specifically, users paid more attention to areas of the videos with spatial audio and had higher pupil dilation when watching videos with third-order ambisonics. The study also found that users' gaze fixations were more evenly distributed when watching videos with spatial audio. These findings have important implications for the development of effective techniques for optimizing processing, encoding, distributing, and rendering content in VR and 360° videos with spatialized audio. The study also suggests that the use of spatial audio improves visual attention and enhances over all user experience and engagement in immersive experiences. CCS CONCEPTS • Information Systems -> Multimedia Streaming