ClassComet: Exploring and Designing AI-generated Danmaku in Educational Videos to Enhance Online Learning

Danmaku, users' live comments synchronized with, and overlaying on videos, has recently shown potential in promoting online video-based learning.However, user-generated danmaku can be scarce-especially in newer or less viewed videos-and its quality is unpredictable, limiting its educational impact.This paper explores how large multimodal models (LMM) can be leveraged to automatically generate effective, high-quality danmaku.We first conducted a formative study to identify the desirable characteristics of contentand emotion-related danmaku in educational videos.Based on the obtained insights, we developed ClassComet, an educational video platform with novel LMM-driven techniques for generating relevant types of danmaku to enhance video-based learning.Through user studies, we examined the quality of generated danmaku and their influence on learning experiences.The results indicate that our generated danmaku is comparable to human-created ones, and videos with both content-and emotion-related danmaku showed significant improvement in viewers' engagement and learning outcome. CCS Concepts