MobileNetV2-Transformer Hybrid Architecture for Effective Video Frame Embeddings Prediction

In the realm of video processing and analysis, accurate prediction of future frames is crucial in applications like video compression, anomaly detection and augmented reality. This paper introduces a novel approach that leverages the powerful feature extraction capabilities of MobileNetV2 combined with the temporal sequence modeling strengths of Transformer models to predict the embeddings of following video frames. More specifically, the paper first employs the MobileNetV2 architecture to extract high-dimensional embeddings from individual frames, capitalizing on its efficiency and robustness in handling image data. These embeddings serve as a compact representation of the visual content of each frame. Then, a Transformer model is utilized to analyze the sequential nature of those embeddings, aiming to predict the embeddings of following frames. The Transformer’s multi-head self-attention mechanism allows for an enhanced understanding of the temporal dynamics of the video, facilitating more accurate predictions compared with traditional methods. The paper later evaluates the method on test video datasets. The results indicate that the model needs further development such as incorporating a greater variety and quantity of experimental data. The method involved in the paper opens new avenues for real-time video analysis applications with the combination of Convolutional Neural Networks (CNNs) and Transformer models. The implications of this research extend to improve the performance of video-based systems across various domains, possibly indicating the potential of integrated architectures in advancing video analytic techniques.