Merged Spatial Temporal Deep Learning Based Content Based Video Retrieval

Imagine searching for a specific scene in a movie, not by remembering its title or actors, but by describing the action itself. This is the essence of content-based video retrieval (CBVR), a technique that searches for a video based on what’s inside it, rather than relying solely on manually assigned labels. Unlike traditional methods, which can be time-consuming, error-prone, and struggle with vast datasets, CBVR offers a more efficient and accurate approach.Our proposed system leverages the strong capability of deep learning, a subset of artificial intelligence, to analyze videos and extract their key characteristics. This process occurs in two stages: offline and online. Through the first stage, important features are extracted from all videos in the dataset and stored for future use. When a user submits a query video, its features are extracted in real-time (online) and compared to the stored features of all videos. The videos with features most similar to the query, essentially those with the “closest match,” are then presented to the user.To capture the full essence of a video, our system employs a two-stream neural network architecture. This innovative approach allows us to extract both temporal features, which capture the changes and motion patterns within the video (think: someone running or jumping), and spatial features, which pivot about the static visual content of each individual frame (think: the objects and scene depicted).By utilizing a pre-trained neural network called ResNet-60, our system benefits from existing knowledge and can efficiently extract meaningful features from videos. To evaluate its effectiveness, we tested our system on the UCF101 dataset, a widely used benchmark consisting of 101 categorized videos. Our approach obtained accuracy 93,7% for top 5 retrieval and 95.95% for top 10 retrieval. The outcomes illustrate that our approach obtains superior accuracy compared to other state-of-the-art video retrieval methods.