Automated Dataset Construction for Composed Video Retrieval in Soccer

Composed Video Retrieval (CoVR) enables flexible video search by retrieving a target video that reflects a specified modification to a query video. The triplet datasets—consisting of query videos, query text, and target videos—required for model training have been collected manually. Recent studies have explored automatic construction of training triplets for CoVR; however, most existing approaches rely heavily on caption similarity. This limitation is particularly problematic in soccer videos, where identical or highly similar captions can correspond to visually distinct situations, making it difficult to construct triplets with appropriate relationships. To address this issue, this paper proposes a multimodal triplet construction framework specialized for soccer videos. The key idea is to explicitly incorporate visual similarity alongside textual similarity. Specifically, candidate target videos are selected by combining visual similarity with commentary caption filtering, enabling the identification of videos that are visually similar yet semantically different. The semantic difference between videos is then generated as query text using a large language model (LLM) without manual annotation. Furthermore, a multimodal large language model (MLLM) is introduced to estimate whether the generated modification is visually and semantically consistent with the video pair. Rather than replacing human verification, this step provides an automated screening signal to identify potentially unreliable triplets. The experiments show that the proposed framework automatically constructs triplets with reasonable validity under limited human validation. These results demonstrate the potential of scalable triplet construction for CoVR in soccer videos.