A SwinBERT framework with temporal windowed cross attention for video captioning for visual language intelligence

Abstract Video captioning serves as a tool which enables computers to understand visual content and convert video footage into written language. The current CNN-RNN video captioning system encounters difficulties when it tries to show both term timelines and visual relationships. Traditional CNN–RNN architectures often struggle to capture long-range temporal dependencies and contextual relationships in video sequences. The research introduces video captioning by utilizing SwinBERT. The proposed research utilizes Swin Transformer for extracting features in videos. The research also uses BERT decoder for captioning. The proposed system generates better captions through its combination of semantic features, which produce natural language output that matches the appropriate context. The proposed system generates captions for reading smoothly and in relation to the context around them. The proposed system demonstrates reasonable caption generation performance on benchmark datasets including MSVD and MSR-VTT, while maintaining computational efficiency suitable for near real-time applications. The use of the GPU can speed up the captioning process. The system operates through real-time applications. The research provides valuable insights which enhance video understanding capabilities for video summarization and assistive technologies.