Towards Reliable Text-Based Video Search: Evaluating Open-Vocabulary Detectors and VLM Verification

Open-vocabulary detectors let a user search video for any object by typing its name, and small versions run on a laptop graphics processing unit (GPU). We ask how far such a search can be trusted when it reports an object, and how far its compute cost can be reduced before it cannot. By compute cost we mean the processing time and GPU memory needed per frame, which can be reduced by using a smaller model, a lower input resolution or fewer frames. We evaluate YOLO-World v2 on 1,972 videos (88,066 frames) of the OVT-B benchmark on a 4 GB laptop GPU, using search words frozen before any detector run: words for objects present in the video, unrelated words, and lookalike words chosen with WordNet and CLIP (Contrastive Language-Image Pre-training). Unrelated words rarely fire (1.1%), but lookalike words often do (23.9%), almost always on a real object of another kind. None of the cost reductions, nor a stricter threshold, removes these errors. Querying all words at once, as benchmarks usually do, hides about a third of these false alarms compared with searching one word at a time. We then add a training-free verification step: a 2-billion-parameter vision-language model (VLM) on the same GPU answers a yes/no question about each detection. It lowers lookalike false alarms from about 24% to about 8% in all six settings, roughly twice the reduction of a stricter threshold at the same loss of real detections. Preprint. Under review at Journal of Big Data.