QANet: quaternion attention network for video summarization

With the rapid increase in video data, automatic video summarization has become paramount in video processing and content analysis. Video summarization aims to generate a concise version of a video that highlights only the most important content, allowing viewers to quickly understand the video’s essence without watching it in full. Although existing methods for automatic video summarization have achieved notable success, they still face challenges such as high computational costs. To address this, we propose a quaternion attention network for video summarization, designed to be both lightweight and efficient. The proposed approach comprises three main components: spatial feature extraction (SFE), learnable frame embedding (LFE) module, and quaternion temporal attention (QTA) module. The SFE module extracts spatial features from each frame, whereas the LFE module generates learned embeddings for the frames. These embeddings are then fed into the QTA module, which captures temporal inter-frame correlations by leveraging quaternion space representations. The proposed approach reduces computational requirements and memory usage while achieving state-of-the-art performance on the SumMe and TVSum datasets.