Key Frame Extraction Based on Sparse Coding with Multiple Features Fusion

Key frame extraction based on sparse coding can present an entire video with a small number of key frames and reduce the redundancy of the video. Currently, key frame extraction techniques mainly rely primarily on single-modality detection. However, single-modal information may not fully reflect the content of the video. For example, relying solely on visual information may miss some important auditory cues. Fusing multi-modal information can provide a more comprehensive perspective and ensure that the extracted key frames are more representative. Therefore, this paper proposes an audio-visual feature fusion method for efficient feature extraction and integrates sparse representation for key frame extraction. By combining features from both video frames and audio, we maximize the utilization of multi-modal information contained in the video. The proposed approach enhances the capability of sparse representation in key frame extraction, facilitating a more robust multimedia content analysis. The experimental results on the SumMe dataset confirm the effectiveness of the proposed approach. The proposed multi-modal key frame extraction can achieve a maximum improvement of up to 8.7% compared to single-modal.