TVMTrailer: A Text-Video-Music AIGC Framework for Film Trailer Generation

In the ever-evolving landscape of media and entertainment, where trends like short-form video content and new media platforms reign supreme, the need for innovative approaches across various industries becomes increasingly apparent. One such industry deeply impacted by these shifts is the film industry, where the creation and dissemination of film trailers stand as pivotal promotional strategies. Making a film trailer by hand is time consuming. However, the emerging artificial intelligence generated content (AIGC) technique has shown significant potential to enhance the efficiency. This article presents a novel model named TVMTrailer, which consists of a text-video generation network (TVGNet) and a video-music generation network (VMGNet). TVGNet employs an encoder–decoder framework, utilizing movie footage and synopses to generate movie trailers. Besides, VMGNet is proposed to generate sound track of our trailer. It combines video and audio features, and uses a transformer model for associative learning to adaptively generate audio clips with features, such as emotion, rhythm and beat. The effectiveness of TVMTrailer is demonstrated through experiment conducted on the proposed dataset and a comprehensive collection of over two thousand video-audio pairs from classic movies.