ColorVideoVDP-ML: Visual Difference Predictor with a neural regressor for image and video quality prediction

ColorVideoVDP-ML is a full-reference image and video quality metric, which combines a visual difference predictor with a machine learning regressor. It employs ColorVideoVDP as an encoder of visual information and visible differences, and a neural architecture as a regressor of the final quality scores. It benefits from low-level vision models of contrast sensitivity and masking, found in ColorVideoVDP, as well as its ability to handle both SDR and HDR content, and to adapt to arbitrary viewing conditions, display resolution, and size. Two neural regressors, one based on MLPs and one on Transformers, are trained on multiple video and image quality datasets, all scaled in the Just-Objectionable-Difference (JOD) units. The regressors let us account for higher-level factors influencing perceived image quality, such as saliency or texture differences, and provide a way to fine-tune and specialize the metric in handling a specific type of distortions. The code for the metric can be found at https://github.com/gfxdisp/ColorVideoVDP.