A hybrid video compression framework using lossless compression and reference-guided restoration network

In this paper, we propose a new lightweight hybrid video codec consisting of a conventional video codec (HEVC or VVC), a lossless image codec, and our new restoration network. The encoder is composed of a conventional video encoder and a lossless image encoder. It transmits a lossy-compressed video bitstream along with a losslessly compressed reference frame. The decoder is constructed with corresponding video/image decoders and a new restoration network, which enhances the compressed video in two-step processes. The first step involves using a network that has been trained with a video dataset to restore the details that are lost by the conventional encoder. After this, we enhance the video quality by using a reference image that is a losslessly compressed video frame. The reference image provides video-specific information, which can be utilized to better restore the details of a compressed video. Experimental results show that the overall coding gain is comparable to recent top-tier neural codecs while requiring much less encoding time and lower complexity. Our code is available at https://github.com/myideaisgood/hybrid_video_compression_rhee.