High Accuracy Rate Control for Neural Video Coding Based on Rate-Distortion Modeling

In recent years, rate control (RC) for neural video coding (NVC) has become an active research area. However, existing RC methods in NVC neglect the actual rate-distortion (<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R-D</i>) characteristics and lack dedicated optimization strategies for intra and inter modes, leading to significant bit rate errors. To address these issues, we propose a high accuracy RC method for NVC based on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R-D</i> modeling, which integrates intra frame RC, inter frame RC and bit allocation. Specifically, the rate-quantization parameter (<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R-Q</i>) model and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R-D</i> model are established for both intra frame and inter frame in NVC. To derive the model parameters, intra frame parameters are estimated using high dimensional features, while inter frame parameters are derived using gradient descent based model update methods. Based on the proposed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R-Q</i> model, intra frame and inter frame RC methods are proposed to determine the quantization parameters (QP). Meanwhile, a bit allocation method is developed based on the derived <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R-D</i> models to allocate bits for the intra frame and inter frame. Extensive experiments demonstrate that, benefiting from the accurate <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R-Q</i> models derived by the proposed approach, highly accurate RC is achieved with only 0.56% average bit rate error. Compared with other methods, the proposed method reduces the average bit rate error by more than 4.18%, and achieves over 8.94% Bjøntegaard Delta Rate savings.