Efficient Integration of ASR with Large Language Models to Enhance Video Search at Scale

Currently, video is the most consumed content on the internet, with an increasing amount of information and knowledge shared as videos. This has led to a significant rise in video search requests. On Bilibili, one of China's largest knowledge-based video platforms, over half of daily users search for videos. Recent advancements in multimodal embeddings and text-to-video retrieval have shown promise, yet large-scale systems require search results within seconds. The high computational costs of image-based models have limited their scalability in video retrieval, with text-based searches remaining dominant and many video contents under-indexed.