Unlocking the secrets of user engagement: the role of multimodal information and sentiment signals in AI agent design

Purpose This study investigates how the information and sentiment of content conveyed by AI agents on digital platforms influence user engagement with multimodal content. Analyzing data on 2,238 agents from the character AI system, we show how text and images activate distinct information-processing channels and jointly shape user interactions. Design/methodology/approach We analyze multimodal data from AI agents using structural topic modeling (STM), VADER sentiment analysis, image information entropy and a ResNet-50 deep-learning model. Grounded in dual coding theory (DCT) and the elaboration likelihood model (ELM), we focus on two attributes – informational richness and sentiment polarity – for both text and images, and test their impacts on users’ online engagement behavior. Findings Text information shows an inverted-U relationship with user engagement. Image visual complexity moderates the effect of textual informational richness. Although aggregate sentiment does not significantly predict engagement, image sentiment amplifies the effect of text sentiment. Images with low emotional intensity create an “emotional vacuum” that increases engagement when paired with positive textual sentiment. Originality/value By integrating DCT and ELM, this study offers a new framework for explaining the behavioral effects of multimodal content. It also introduces a method for quantifying the informational and affective attributes of text–image pairs. The findings provide actionable guidance for optimizing digital marketing content and the design of AI-driven conversational agents.