Nonlinear Modifications of Transform Coefficients in VVC Intra Coding

With the emergence of the Versatile Video Coding standard (VVC), novel coding tools like matrix-based intra prediction and low-frequency non-separable transforms have been developed based on data-driven optimization methods. Motivated by the growing compression efficiency of learned nonlinear transforms in image coding, we incorporate a neural-network-based coding tool into the transform coding stage of VVC. First, a nonlinear update of the transform coefficients is applied which aims at decreasing the expected bitrate cost. Then, on the decoder side, a filter is applied before the synthesis transform to improve the reconstruction quality. These networks are jointly trained and have been added to the rate-distortion optimized quantization process. The integration of these approaches into the VTM software leads to bitrate savings between 0.94% and 2.12% in terms of the Bjøntegaard-Delta rate. We furthermore conducted multiple experiments on reducing the memory complexity of the networks by exploiting structural similarities of the intra prediction modes and block symmetries. As a result, we demonstrate that the number of distinct networks can be reduced without significantly diminishing the coding gain of the tool. With memory usage reduced by 90%, we achieve bitrate savings between 1.07% and 1.94%.