Challenges in video-text retrieval via transparent alignment

Video-text retrieval is a crucial task in understanding multimedia content. Current applications often face challenges related to limited dataset diversity and high inference costs. Unlike image-text datasets, which tend to be larger and more diverse, paired video-text datasets are usually smaller and less varied. To address these issues, we propose a method called Explainable Video-Text Alignment (EVT), which utilizes pretrained encoders and trains lightweight alignment matrices. EVT is a transparent and efficient retrieval framework that consists of three stages: (1) preprocessing, (2) clustering, and (3) alignment. This framework provides interpretable intermediate results by framing the retrieval process as a concept-matching task. With a plug-and-play design, EVT maintains generalizability while requiring only 3% of the typical trainable parameters. In addition, we visualize the cases where the method fails to identify areas for further improvement.