Towards Content-Optimized Selection of Video Spatiotemporal Resolution for HTTP Adaptive Video Streaming

In adaptive video streaming systems, selecting optimal spatial and temporal resolutions for a given bitrate is crucial when encoding various video representations on the server side. Previous research has primarily focused on adjusting spatial resolution, leaving optimization in the time-space domain as an open research question. The research aims to identify the optimal spatiotemporal resolution for a given bitrate, contingent on the complexity of the video sequence. Given the extensive requirements of such research, this paper presents preliminary findings based on five sequences of differing complexity. For low-complexity sequences, the optimal approach is to first reduce the temporal resolution from 120 fps to 25 fps, followed by a reduction in spatial resolution. Conversely, for sequences with medium and high complexity, it is preferable to decrease the spatial resolution first and then the temporal resolution. The appropriate bitrates for these resolution adjustments are also influenced by the complexity of the video signal. The encoded sequences, along with their objective and subjective quality evaluations will serve as a foundation for future AI-based modeling efforts.