UNIFIED SPATIO-TEMPORAL VISION TRANSFORMER FRAMEWORK FOR REAL-TIME MULTI-MODAL VIDEO UNDERSTANDING AND EVENT PREDICTION LEVERAGING CROSS-ATTENTION FUSION AND SELF-SUPERVISED LEARNING FOR COMPLEX SCENE INTERPRETATION IN DYNAMIC ENVIRONMENTS

The rapid advancement of deep learning architectures has revolutionized video understanding, especially in dynamic, real-time applications such as autonomous vehicles, surveillance systems, and sports analytics. However, current models often struggle to jointly model spatial-temporal dynamics and cross-modal dependencies, leading to suboptimal performance in complex scenes. This paper presents a unified spatio-temporal vision transformer framework that integrates self-supervised learning with cross-attention fusion for multi-modal video understanding and event prediction. The proposed system employs a transformer-based encoder to capture spatio-temporal relationships, enhanced through fusion mechanisms that process RGB frames, motion cues, and audio signals. Self-supervised learning techniques ensure better generalization in real-world scenarios with limited annotations. Extensive benchmarking against state-of-the-art models demonstrates superior accuracy, robustness, and interpretability. This research provides a scalable and efficient solution for complex scene understanding, capable of real-time deployment in edge environments. Performance metrics show a substantial increase in predictive accuracy, supported by reduced training loss and improved F1 scores across test datasets.