Multi-Branch Collaborative Learning Network for Video Quality Assessment in Industrial Video Search

Video Quality Assessment (VQA) is a crucial component of broadscale video retrieval systems. Its goal is to accurately identify various quality issues in videos, thereby encouraging the video retrieval system to prioritize high-quality videos. In large-scale industrial video retrieval systems, we formulate the characteristics of low-quality videos into four categories: visual-related low-level quality problems such as mosaics and black boxes, textual-related low-level quality problems caused by video title and Optical Character Recognition (OCR) content, as well as semantic-level frame incoherence and frame-text mismatch caused by emerging AI-generated videos. These kinds of low-quality videos, which are widely present in industrial environments, have been overlooked in academic research before, and accurately identifying them is very challenging. In this paper, we introduce a Multi-Branch Collaborative learning Network (MBCN) to tackle the above issues. We carefully design four assessment branches for MBCN to adapt to the above four kinds of issues for industrial video retrieval systems. After obtaining independent scores for each branch, we perform a weighted aggregation of the various branches to dynamically address video quality issues in different scenarios with a squeeze-and-excitation mechanism. Finally, we integrate point-wise and pair-wise optimization objectives to ensure the predicted scores are stable and fall into a reasonable range. To demonstrate the effectiveness of our proposed MBCN, we conduct extensive offline and online experiments in a world-level video search engine. The experimental results show that due to the powerful ability of MBCN to identify video quality issues, the ranking ability of the video retrieval system has been significantly improved. We also conduct a series of detailed experimental analyses to verify that all four evaluation branches play a positive role. Besides that, for emerging low-quality AI-generated videos, the recognition accuracy of MBCN also improves significantly compared to the baseline.