A Multidimensional Media Adaptation Framework for Live Holographic Communication

The rapidly increasing popularity of immersive multimedia services such as live holographic communication represents the future trend of extended reality (XR) applications. However, the realization of such immersive and interactive experiences is limited by the lack of fundamental understanding of how different user behaviours and environmental factors jointly affect the overall quality of experience (QoE). In particular, compared with the media adaptation mechanisms applied in conventional video applications, considerably more independent factors may influence user QoE in these applications, including both human- and network-related factors. In this paper, we investigate the fundamental design principles of dynamic media adaptation methods for live holographic communication by holistically considering these factors. Specifically, a machine learning-based scheme is introduced to facilitate intelligent adaptation of both the frame quality (resolution level) and frame rate according to specific contexts, such as the user intent/behaviour (including object motion patterns and user movements) and real-time network conditions. Extensive real-world experiments are conducted to assess the feasibility and performance of the proposed method, and comparisons with state-of-the-art methods are performed. The results indicate that the proposed approach can effectively satisfy user intent, with increased user QoE.