AdaVQA: a semantic and quality-aware video quality assessment method

Video quality assessment (VQA) has become increasingly important for video service providers due to the explosive growth of video traffic. It plays an indispensable role in balancing compressed video quality with compression rates. In recent years, the rapid advancement of deep neural networks has led to effective strategies that utilize Convolutional Neural Networks or Transformer to extract video embeddings and generate quality scores through Multilayer Perceptron. However, the labor-intensive and time-consuming process of annotating video quality has caused existing VQA datasets to be relatively small in scale. This limitation makes the VQA task highly reliant on pre-trained parameters on classification task, which is suboptimal due to the gap between the parameters trained on the classification task and the requirements of the VQA task. In this paper, we propose a solution to bridge this gap by introducing an Adapter at each network layer, enhancing quality awareness while leveraging pre-trained semantics classification parameters, named AdaVQA. Extensive experiments on four mainstream VQA datasets demonstrate significant performance improvements over traditional methods. Ablation studies further confirm the effectiveness of our method in aligning semantic-aware embeddings from classification model to quality-aware VQA task.