Decoupled Distillation-Based Emotion-Oriented Trailer Generation System

Multimodal emotion recognition(MER) refers to the process of inferring emotions expressed by individuals using text, audio, and image training samples, Currently, many multimodal models have achieved state-of-the-art performance on sentiment analysis datasets. However, current research mainly focuses on training and recognition on datasets, while neglecting its application in the industrial sector. This article proposes a video synthesis algorithm, used to generate movie trailers, Ultimately applied to future TV recommendation systems. Firstly, the CMU MOSI dataset is used to train a multimodal model. The input video is split into shots and sampled at equal intervals on a frame by frame basis. Then, optical character recognition (OCR) technology is used to extract subtitles, and the extracted subtitles, audio, and video frames are input into the model, and emotional scores are given to each shot. Finally, the desired movie emotional shots can be adjusted by setting emotional thresholds, and combine these candidate shots into a complete trailer. The experimental results show that the output movie trailer can meet the emotional needs of users, which is of great significance for the application of artificial intelligence in video editing.