Video-Wise Just-Noticeable Distortion Prediction Model for Video Compression with a Spatial–Temporal Network

Just-Noticeable Difference (JND) in an image/video refers to the maximum difference that the human visual system cannot perceive, which has been widely applied in perception-guided image/video compression. In this work, we propose a Binary Decision-based Video-Wise Just-Noticeable Difference Prediction Method (BD-VW-JND-PM) with deep learning. Firstly, we model the VW-JND prediction problem as a binary decision process to reduce the inferring complexity. Then, we propose a Perceptually Lossy/Lossless Predictor for Compressed Video (PLLP-CV) to identify whether the distortion can be perceived or not. In the PLLP-CV, a Spatial–Temporal Network-based Perceptually Lossy/Lossless predictor (ST-Network-PLLP) is proposed for key frames by learning the spatial and temporal distortion features, and a threshold-based integration strategy is proposed to obtain the final results. Experimental results evaluated on the VideoSet database show that the mean prediction accuracy of PLLP-CV is about 95.6%, and the mean JND prediction error is 1.46 in QP and 0.74 in Peak-to-Noise Ratio (PSNR), which achieve 15% and 14.9% improvements, respectively.