Dynamic reference camera selection for free-viewpoint video multicast streaming

Abstract Free-viewpoint video (FVV) is an advantageous technology that allows users to interactively change their viewpoint or perspective when viewing a video or a scene. It provides a three-dimensional, dynamic, and immersive viewing experience, allowing users to explore a video scene from different angles and viewpoints as if they were present within the scene. FVV requires transmitting multiple video streams with depth information for different perspectives simultaneously. This places a significant burden on network bandwidth and infrastructure, leading to congestion and potential service degradation. The paper investigates dynamic reference camera grouping approaches, which are based on the users’ specific viewpoint needs. The primary goal of these methods is to minimize the network bandwidth necessary for transmitting reference color and depth camera sequences, thus enabling the provision of interactive FVV services in networks with limited resources. To achieve a reduction in the number of essential camera streams, we introduce dynamic reference camera selection methods that effectively identify shared reference camera pairs capable of serving multiple users for their individual viewpoint synthesis requirements. We found that among the investigated methods, DBSAN performance was the most balanced among the investigated algorithms, achieving 10–20% network load decrease, depending on the network complexity.