Consistent Quality in Video Coding: A Perceptual RDO and Neural Network-Based QP Prediction Approach

Video coding has been greatly growing in the recent four decades with a number of standards such as H.264/AVC, HEVC or VVC which have opened a new era of multimedia applications such as video conference, video surveillance, broadcasting, and E-learning. However, to achieve high compression performance, video coding standard like HEVC typically suffers from the quality fluctuation which may directly affect the quality of experiences (QoE). To cope with this problem, we propose a novel video coding framework targeting two purposes, the consistent quality and the compression efficiency. First, to achieve the perceptual quality of video frame, we propose to use the VMAF (video multi-methods assessment fusion) metric as the distortion measurement instead of PSNR or SSIM in literatures due to its high correlation with the human perceptual system. Second, we propose a novel VMAF-QP (quantization parameter) neural network that learns how QP can be adjusted to achieve a certain VMAF grade. Following the estimated QP, we propose a RDO process to guarantee the compression efficiency of the proposed codec while maintaining the decoded video with quality consistency. In this process, the Rate-QP and Distortion-QP models are introduced and optimized through a number of statistical analyses. Finally, we integrate the proposed consistent quality video coding (CQVC) framework into the HEVC standard. Experimental results reveal that the proposed video coding framework achieves a small quality fluctuation while still guarantying the high compression efficiency, notably 4.5% Bitrate reduction when compared to the standard HEVC.