Multimodal understanding of human values in videos: A benchmark dataset and PLM-based method

Multimodal content has become the mainstream communication medium in video sharing platforms such as TikTok and Twitter, containing rich values information. Understanding human values is of great significance to multimodal content analysis and can be applied to downstream tasks such as recommendation systems and value alignment. However, current studies on human values mainly focus on text and lack a multimodal perspective. In this work, we present a new multimodal human values video dataset called VVALUES, which contains 5,104 annotated videos along with their titles. The dataset is labeled with coarse-grained polarity tags of positive and neutral, and fine-grained tags including 13 classes of value vocabulary. Based on VVALUES, we further develop a pre-trained language model (PLM)-based multimodal method adopting a dual-transformer variant for value recognition, MMVR. Extensive experiments demonstrate that our method significantly improves the performance of understanding values in videos. To the best of our knowledge, we are the first to try to incorporate human values in video understanding , and VVALUES is the first multimodal video dataset for human values. • We are the first to try to incorporate human values in video understanding. • We present a new multimodal human values video dataset called VVALUES. • We develop a PLM-based multimodal value recognition method, MMVR.