Most Significant Impact on Consumer Engagement: An Analytical Framework for the Multimodal Content of Short Video Advertisements

The increasing popularity of short videos has presented sellers with fresh opportunities to craft video advertisements that incorporate diverse modal information, with each modality potentially having a different influence on consumer engagement. Understanding which information is most important in attracting consumers can provide theoretical support to researchers. However, the dimensionality of the multimodal features of short video advertisements is often higher than the available data, posing specific difficulties in data analysis. Therefore, designing a multimodal analysis framework is needed to comprehensively extract and reduce the dimensionality of the different modal features of short video advertisements, thus analyzing which modal features are more important for consumer engagement. In this study, we chose TikTok as the research subject, and employed deep learning and machine learning techniques to extract features from short video advertisements, encompassing visual, acoustic, title, and speech text features. Subsequently, we introduced a method based on mixed-regularization sparse representation to select variables. Ultimately, we utilized multiblock partial least squares regression to regress the selected variables alongside additional scalar variables to calculate the block importance. The empirical analysis results indicate that visual and speech text features are the key factors influencing consumer engagement, providing theoretical support for subsequent research and offering practical insights for marketers.