Towards An Improved Video RAG Workflow With Orchestration Support in A Visual Data Management System

Video Retrieval-Augmented Generation (RAG) Workflows augment user query contexts with stored video contexts to guide a Large Language Model (LLM) in providing more context-relevant answers. The challenge is to design a workflow where the video contexts can be generated with practical latency values. It is also important to ensure that the resources available to run the workflows are not under-utilized whenever possible. In this paper, we propose a video RAG workflow that utilizes the Visual Data Management System (VDMS) interfaced with the Kubernetes (K8s) orchestration framework to design an enhanced video RAG workflow (VRAG) for faster video context generation in the pre-processing phase and a faster response generation with no impact on response accuracy. Our experiments showcase that compared to Conventional RAG(CRAG) workflows, VRAG reduces the pre-processing latency by a minimum of 10%, which decreases manifold with parallelization and the response latency by a minimum of 70% to as high as 97% with further parallelization.