No-reference image quality assessment using a deep residual network

Many algorithms have been developed for no-reference image quality assessment (NR-IQA). While various problems will arise in the process of designing the network structure, such as feature loss, gradient disappearance and insufficient generalization ability and so on. This work introduces a deep multi-scale residual CNN for NR-IQA to address the issues. The significant improvement in our model's effectiveness is attributed to the optimization of traditional residual networks by eliminating unnecessary modules. The residual structure to form a deep network, which consists of several residual blocks with skip connections. Residual block contains convolution kernels of different sizes to adaptively extract image features, which are merged with each other to obtain effective image information. Then, use the output of each multiscale residual block as the hierarchical feature of global fusion, so as to retain as much image information as possible for a final quality prediction. An auto-encoder based method is introduced to reduce the dimension of local feature in multiscale residual block to improve training efficiency. Experiments show that the model is superior to the existing state-of-the-art methods in the prediction of various distortion types in the LIVE dataset. The results of cross dataset validation on the TID2013 and CIQA datasets also reflect the better performance of the model. Finally, in order to prove the generalization ability of the model, we generate Gradient Class Activation Maps (Grad-CAM) to compare with VGG16 and Resnet18 for feature visualization. The results of our experiments reveal that our model possesses strong robustness and excellent generalization capabilities.