A Novel Framework for Video Saliency Feature Detection Using Deep Learning

Video saliency detection has growing importance in fields such as video segmentation, robotic activity, autonomous driving, video captioning, and video compression. At the same time, the task is often complicated by factors such as motion blur, occlusions, and changes in the scene, along with the more typical challenges of human visual perception (HVS). In addition, many existing approaches suffer from error propagation due to poor detection of low level visual elements. In this work, we present a new framework to address the problem of video saliency detection using deep learning techniques. The framework features U2Net for multi-scale subfeature extraction and uses saliency maps refinement based on adaptive attention techniques. Furthermore, a novel hybrid spatialtemporal contrast computation technique is used to improve robustness to motion diversity. To improve understanding and user-friendly interaction, three background colors are implemented: transparent, black, and white. The real time video processing is enabled by a frontend built on Python, HTML, and CSS with Flask. The data collected during the experiment indicates that this approach improves the discrimination of salient regions while requiring less effort.