Improving Watching Comfort of Live Free Viewpoint Video by Catch-Up Playback

This paper proposes a method to improve watching comfort by catch-up playback to fill the time gap that occurs when observing free viewpoint video (FVV) in live. Live FVV in live sports events becomes to attract significant popularity and attention. Developing interfaces for observing FVVs is a critical research issue to further provide viewers with comfortable observation. FVV users can freely switch viewpoints during live streaming without pausing playback. However, simultaneous video playback and viewpoint switching make it difficult to observe key moments from multiple angles. So, it is necessary to temporarily pause playback when switching viewpoints to observe key moments such as a soccer shot or a baseball swing from various directions. On the other hand, pausing playback causes a time gap between the progress of live event and the video play. The sense of sharing and unity in real time, as well as the emotion and excitement of each moment, is a major significance of live events. The video delay is a serious problem that detracts the significance of live events. To address this issue, we propose a catch-up playback method to recover delays caused by the observation of key moments. This research aims to recover the delay as quickly as possible while maintaining the watching comfort of the video. This paper proposes three types of catch-up playback methods. The first method is Skip Playback, where the video instantly skips forward to the live point immediately after resumption. The second one is Constant-Speed Playback, where the video resumes at a constant high-speed playback until catching up to the live point. The third one is Accelerated-Decelerated Playback, where the video gradually accelerates towards the live point and then decelerates back to normal speed. These methods, which dynamically adjust the playback speed according to the delay time, aims to both maintain watching comfort and recover the delay. Subjective evaluation experiments to verify the effectiveness of each method are conducted. In the experiment, we prepare a FVV of a soccer game that had been previously recorded, and apply “video stop”, “viewpoint switching”, and “the three types of catch-up playback” processes to the video. After participants viewed the three types of videos, SD (Semantic Differential) questionnaire surveys with 12 items are conducted to evaluate impressions of each method. Factor analysis reveal that participants evaluate the videos on two evaluation axes: comfort and dynamism. Results show that Constant-Speed Playback achieves the highest comfort ratings. Constant-Speed Playback significantly outperforms the other two methods, particularly in terms of "ease of viewing." On the other hand, Skip Playback receives the lowest comfort ratings. Accelerated-Decelerated Playback is rated significantly lower than Constant-Speed Playback in terms of "ease of viewing" and "pleasantness." These results suggest that Constant-Speed Playback recovers the delay in live FVV without compromising watching comfort compared to Skip Playback and Accelerated-Decelerated Playback. Additionally, it can be considered that continuous speed changes during catch-up playback negatively impact watching comfort.