Visual Keyword Spotting with Multi-Encoder for MAVSR 2025

This paper systematically describes the Fosafer system designed for the Mandarin Audio-Visual Speech Recognition (MAVSR) Challenge 2025 Track 2. The purpose of Track 2 is to evaluate the performance of visual speech analysis systems in identifying whether a speaker has pronounced specific keywords in silent video sequences, also known as visual keyword spotting (VKWS). In this paper, we propose to improve VKWS with a video augmentation method and visual multi-encoder. Specifically, we first propose a simple but effective video augmentation method to fully utilize the scarce video data in VKWS. Secondly, visual multi-encoders like Transformer and Conformer for lip motion feature extraction are implied. Finally, we adopt the strategy of model fusion to further enhance model performance. Ablation experiments show that each module of our proposed system has a different level of enhancement to the visual keyword spotting results. The proposed method ranked first in Track 2 of the MAVSR 2025 Challenge, achieving a Mean Average Precision (mAP) of $36.226 \%$ on the test dataset.